*********************END OPTIONAL*********************
| [Choose Language] This determines the language of the installer and picks a keyboard. This installation has only been tested with English - English |
| [Choose country or region] Choose what is appropriate |
| Unplug the Ethernet cable, we want DHCP to fail. |
| [Select a keyboard layout] American English selects a standard qwerty keyboard |
| [Module needed by your ethernet card] |
Look through the list; the majority of cards that are supported will NOT be on the list, this is a list of somewhat obsolete or possibly bleeding edge cards, not the majority. If it does not find your card, try another NIC Card. The machine I am building right now has an old 3Com ISA PnP 3c509, and it's not working, possibly because PnP may not be supported this early in the game, or because there is a bug in the installer. This is going to be an email server, so a reliable NIC card is important. If you have and old ISA NE2000 compatible card you can use the "ne" driver but you will need to know the interrupt and I/O address beforehand. If you have a problem, the fastest way to solve the problem may be to replace the card with another model.
| [!! Configure the network]
Network autoconfiguration failed We wanted this to happen, simply press: [Continue] On the next screen, choose the default of: [Configure network manually] |
| [Configure the network] Plug the Ethernet cable back in (and make sure Num Lock is on!) [IP address:] 111.111.111.111[Netmask:] 255.255.255.x[Gateway:] 333.333.333.333[Name server addresses:] 444.444.444.444 555.555.555.555[Hostname:] sfa[Domain name:] example.com
|
******************************************************
Partition the Hard Drive:
Index******************************************************
The disk partitioning software that comes with this version of the Debian Installer seems to be geared toward novices and as such makes a lot of assumptions in order to make partitioning easy. However, if you want to deviate from what the software provides, it is somewhat cumbersome. At this point you can play with the partitioning software all you like. If you have problems, simply erase the disk and start over.
| [!! Partition disks] Choose [Erase entire disk] [Partitioning scheme:] You are free to choose any of the three partitioning schemes provided but we need at least 1GB of space for each data partition. If you choose the [Desktop machine] or [Multi-user workstation] method of partitioning, ideally you would have 4GB or more for either the /var partition or the /var/spool partition respectively. If you are building this with a 1GB or 2GB drive (not recommended) or simply want to maximize disk space, choose [All files in one partition] For a larger drive choose: [Desktop machine] Then arrow up and change the "Mount point:" of partition "#6 logical" from /home to /var Here is an example of what the finished product could look like: IDE1 master (hda) - 10.0 GB Maxtor 5T010H1 hda1 #1 primary 2.8 GB ext3 / (bootable) (root partition) hda5 #5 logical 353.7 MB swap (swap partition) hda6 #6 logical 6.8 GB ext3 /varThe same drive using and modifying the [Multi-user workstation] partitioning scheme: It took me about 10 minutes of playing with the software to figure out how to modify what the partitioning software came up with, but this will provide a little better performance (due to reduced file fragmentation). You need a 6GB or larger drive and a little patience to do this. Change the mount point of "#1 primary" from / to /boot Change the mount point of "#5 logical" from /usr to / Change the mount point of "#6 logical" from /var to /var/lib (Enter manually) Delete both partitions #9 and #8, then recreate logical partitions #8 and #9 from the free space and change the mount points to what is illustrated below. Each data partition should be at least 1GB as shown. The /var/spool directory is where our mail queues will be, so it would be desirable to make it 3GB or larger. IDE1 master (hda) - 10.0 GB Maxtor 5T010H1 hda1 #1 primary 279.6 MB ext3 /boot (bootable) hda5 #5 logical 3.6 GB ext3 / (root) hda6 #6 logical 1.8 GB ext3 /var/lib hda7 #7 logical 386.6 swap swap hda8 #8 logical 1.0 GB ext3 /var/log hda9 #9 logical 3.0 GB ext3 /var/spoolOnce you have what you like, choose [Finish partitioning and write changes to disk] [Write changes to disk?] [Yes] |
|
[Installing the Debian base system] Wait.... [Install the GRUB boot loader to the master boot record?] If you would like the install the GRUB boot loader choose [Yes] If you would like the install the LILO boot loader [Tab] over and select [Go Back] Then select the 'Install the LILO boot loader...' [Finish the installation] Remove the CD or floppy when prompted, then hit [Continue] This will reboot. |
The Debian Installer has done its job, now it's time for the:
******************************************************
Debian base system configuration:
Index******************************************************
| [Debian base system configuration] - Welcome to your new Debian system! [OK] |
| [Time zone configuration] [Is the hardware clock set to GMT?] [NO] [Select your time zone:] Simply choose what is appropriate. |
| [Password setup] This will ask for root's password and allow you to create a "normal" user and a password for that user. Watch your [Num Lock] status. Use really good passwords and don't forget them. Please add one, and just one, normal user here. If you plan on storing mail locally on this machine (not documented here), or even if you don't, create a user who's main purpose in life might be to hold root's mail. I suggest calling the user myroot or something similar. |
|
Make sure you are connected to the Internet
[Apt configuration] [Archive access method for apt:] Choose [http] then your [Mirror country] then a mirror near you. (mirrors.kernel.org works very well in the US) [HTTP proxy information] (configure if needed, otherwise leave unconfigured) |
| [Debian software selection]
[Choose software to install:] Do not select anything here. Simply [Tab] over and select [Ok] |
This section should not show up, but just in case it does:
| [Configuring console data] IMPORTANT! choose "Don't touch keymap" You chose one earlier whether you knew it or not and choosing any keyboard here may remove the keyboard mapping and you may not be able to get it back without starting the installation over!!!!! |
| [Configuring Exim v4 (exim4-config)]
[General type of mail configuration:] choose [no configuration at this time] [Really leave the mail system unconfigured?] [Yes] [Root and postmaster mail recipient:] The "normal" user we added earlier will display here. This is fine, so simply accept this. Since all mail will be relayed to another server, this setting will actually end up being ignored. However, if you configure your system to store mail locally, all of root's mail will be redirected to this "normal" user's mailbox. This is necessary because you typically cannot access root's mailbox remotely. |
| [Debian base system configuration]
[Setup of your Debian system is complete] [OK] |
Once you get the login prompt, login as root and issue the following command:
apt-get install ntpdate ssh vim gpupg
|
Thanks to the ntpdate program, our system should now have the correct time. Now we need to set our CMOS clock to the same time as our system:
hwclock --systohc
|
Enter the following command:
dpkg-reconfigure locales
|
|
[Configuring locales] You use [PgUp] [PgDn] [up-arrow] [down-arrow] [tab] and [spacebar] to navigate and select. Your ISO-8859-x locale should already be selected. You can simply [Tab] over to [Ok]. The installer software correctly assumed I wanted en_US ISO-8859-1. I suggest you have this also (in addition to others if you require them). If you need to change the locale, or add additional locales, use the [arrow] [spacebar] and [tab] keys. You may wish to add your corresponding UTF-8 locale but a UTF-8 locale should not be used as the default system LANG (set in /etc/environment), SpamAssassin and amavisd-new may have problems if you do. [Which locale should be the default in the system environment?] I suggest you do NOT choose [None], I suggest you choose [en_US] |
We need to make sure we have a keymap file:
ls -l /etc/consoleThis lists the contents of the /etc/console directory.
You should see a file named "boottime.kmap.gz"If you get "Total: 0" then we have no keymap file. If, and only if, we have no keymap file, run the command: dpkg-reconfigure console-dataAnd choose [Select keymap from arch list] Follow the prompts that apply to you and when the program exits check again to see if there is now a file called "boottime.kmap.gz" in the /etc/console
directory.If the file is not there, reboot and try again. We cannot continue until a keymap file is installed. Worst case is we would have to start the installation over again! |
Note that you can use the [up-arrow] key to recall previously entered commands (which can then be edited and executed).
| Once you are back at the shell prompt, reboot the system with [Ctrl][Alt][Delete] |
This is the end of the basic Debian installation.
If you don't feel good about the way things went, or you would like to experiment with one of the other methods of installation, this would be the time to start over from scratch!
******************************************************
PuTTY and additional programs:
Index******************************************************
Now turn the monitor off and head on over to your trusty old Windows computer. We are going to configure every thing else from there!
You should have this document open in a window on your Windows computer because we are going to use a Windows SSH client called PuTTY to operate our spamfilter remotely. I am going to save you a lot of typing because you are going to select text with the mouse, copy it to the clipboard with [Ctrl]+c and then paste it into the PuTTY window with a right-click of the mouse. This will save you a ton of typing.
Download putty.exe from somewhere like: http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html
Place putty.exe on your desktop, open it up, select SSH, input the IP address of the spamfilter then enter a name for your session in the Saved Sessions box. In the Category window, expand Terminal and click on Features. Check the box "Disable application keypad mode". Just below this in the Category window, click on Window, and increase "Lines of scrollback" from 200 to 400. Click on Session (at the top), then Save. the "Disable application keypad mode" enables us to use the numeric keypad when using vi.
When you use PuTTY again, simply double click on the saved session. Make sure you are at a command (shell) prompt before exiting PuTTY. You can log off the PuTTY window by issuing the command 'logout' or 'exit' or [Ctrl]+d.
If you are selecting some text to paste onto the Linux command prompt (the bash shell), you normally should not select the empty line below the text we are selecting. If you do, this has the same effect as hitting the [enter] key when it is pasted into the PuTTY screen. Sometimes this is desirable. If you select only to the end of the text, you will have the opportunity to review what was pasted before you hit [enter]. Make sure the command line is empty before you paste something into it.
If you are editing a configuration file (we will use the vi editor to do that), you may select multiple lines to copy and paste. To insure there is a newline character at the end of each line I suggest extending your selection to the empty line below the text you are selecting. I will provide empty lines below any text that ends up getting pasted into a document we are editing with vi.
Some of the text in this document will have to be edited either before or after you copy and paste it into the PuTTY window. If you have not already done so I recommend you save this document as a text file or an html file and do a search and replace of key items like your domain name and IP address, then use that document to continue. I suggest using WordPad or your favorite plain text HTML editor to edit this file. Do not use a program that will modify the HTML code. If you open this document in WordPad there are instructions at the top of the page. Once your changes have been made, open the saved document in your browser. You will now have your own semi custom document.
Note.
Your [Home] and [End] keys will work when editing a file using vi, but will not work at the command prompt (you can use [Ctrl]+a and [Ctrl]+e for this).
Open up a PuTTY session now and log in as root.
The command prompt at the bash shell will look something like:
sfa:~#
The ~ (tilde) represents our home directory, and because we logged in as root, our home is /root
Try this command:
pwd(which means "print working directory"; it will reply with "/root") |
This would be a good time to also download and install WinSCP. WinSCP is a great GUI file browser that lets you transfer files between your Windows machine and your new Debian box. You can also edit files on your Debian box from your Windows machine using WinSCP's editor.
Important note: The machine is very vulnerable at this point. Any time you are not working on the spamfilter, you should unplug the ethernet cable! This machine should be connected to the Internet only when necessary to configure it. We need to get all our security measures in place before it "goes live". You will need to leave it plugged in to complete the installation however. If you are familiar with editing files on a Linux system, it might be a good idea to jump ahead to "Create Firewall Rules" and then return here to continue.
******************************************************
The 2 minute vi tutorial:
Index******************************************************
We are going to use vi (vim actually) to do most of our editing. Fortunately we only need to learn a few commands to be able to accomplish our tasks. There are 3 operating modes in vi. There is the "Command" mode, the "Write" mode and the "Command line" mode. When you first open a file for editing, you are in Command mode. You change to Write mode by entering the letter "
i",
(short for "insert"). You can edit text pretty much as you would expect in
Write mode. You exit out of Write mode and return to Command mode by hitting
the [Esc] key. There are many commands that can be learned in Command mode but
we only need to learn two more in addition to "i". Those commands
are ":" (a colon) and "/" (a forward slash). The
colon is used to enter the third mode, the Command line mode and the slash
enables the Search command. When you are in Command line mode, you will see a
colon at the bottom of the screen. Here is a list of commands we will use while
in Command line mode:
:q quit (provided you have not made any changes) By the way,
the lower case q is used often in *nix as a way to exit a screen. :q! exits vi and discards changes (great when you trashed the
file and just want to start over!) :wq saves the changes and exits vi (write and quit) :w saves the current changes but does not exit vi (write)And in command mode:
G The capital "G" Goes to the bottom of the page (very handy)
And here is how the Search command works:
/text_to_search_for moves the cursor to the first occurrence of
text_to_search_for
Once the first occurrence of the text we searched for is found, use a lower case 'n' to find the next occurrence.
That's all we need to know for now!
If you would like a cheat sheet for additional commands: http://www.fprintf.net/vimCheatSheet.html and http://amath.colorado.edu/computing/unix/vi/
If you install vim from 'testing', vi may no longer use vim (which is what we actually want), so you may need to point vi to vim:
Run this to see which program vi points to:
ls -l /etc/alternatives/vi
If it says it is pointing to nvi or other program instead of vim (like this): /etc/alternatives/vi -> /usr/bin/nvi Then we need to fix it so it points to (symbolically links to) vim:
mv /etc/alternatives/vi /etc/alternatives/nvi
If you run this again:
ls -l /etc/alternatives/vi
You should see that it now links to vim. |
******************************************************
Verify System Settings:
Index******************************************************
We need to take a look at a few important files to make sure they are accurate.
vi /etc/resolv.confMake sure our domain name is at the top, in the form: search example.com The file should look something like: search example.com nameserver 444.444.444.444 nameserver 555.555.555.555 Repair it if it is not. (Use "i", then edit it). If you made changes, Exit the file with [Esc] : wqIf you did not need to change anything, Exit the file with [Esc] : q
|
vi /etc/hostsThe top of file should look something like: 127.0.0.1 localhost.localdomain localhost 111.111.111.111 sfa.example.com sfaRepair it if it does not (localhost.localdomain is not strictly required). Remember, use "i" to insert. If the hostname "sfa" is listed on both lines, remove it from the 127.0.0.1 line. Since we are here, you might as well add any other hosts you would like our spamfilter to know about. I suggest you (at least) put your internal mail server(s) here. Simply append any other entries to the bottom of the list. If you made changes, Exit the file with [Esc] : wqIf you did not need to change anything, Exit the file with [Esc] : qIf you have a mess on your hands, Exit the file with [Esc] : q! and try again.
|
If you made changes to any of the above files:
rebootlogout |
FYI, to power down the system, the command is:
shutdown -h now
******************************************************
Change apt-get settings:
Index******************************************************
In Red Hat you would use "update -u" to get updates to installed packages. In Debian, you use
apt-get update (to update the local database of available
packages) followed by apt-get upgrade, to install the
latest version of any and all packages it found on our system. This is fine because
we are using the 'stable' version of Debian, but you should not upgrade your system
or install packages indiscriminately, especially if you use any packages from
the 'testing' or 'unstable' branches. This could make stuff stop
working. Fortunately there is something called "Apt-Pinning" that enables us to
prioritize the order of 'stable', 'testing', and 'unstable' software sources. This file
has to be created by us.
The most succinct explanation of this can be found at
http://jaqque.sbih.org/kplug/apt-pinning.html. If you ever use "apt-get upgrade",
I strongly recommend using
apt-get -s upgrade to "simulate" the upgrade process
before you actually upgrade.
This next file is critical to the way our system functions.
vi /etc/apt/preferences |
| Enter this text in the file ("i" to insert) EXACTLY as shown. Yes, you can select the text with your mouse, hit [Ctrl]+c , and then right-click in the vi editor window.
Package: *
Exit the file with [Esc] : wq as usual.
|
I recommend you use
apt-get -s install [package]
before you install any package. It lets you "simulate" what would happen.
If you want a package that is an
'unstable' or 'testing' version, you would have to specifically request the 'unstable'
or 'testing' version or change the priority before you install it (unless the only version
is 'unstable' or 'testing' or your current version is 'unstable' or 'testing'). For example
apt-get -t unstable install [package]. If you use tools like
tasksel, you may have to temporarily change the priority prior to installing a
new set of packages. The most stable situation is to only upgrade to new
packages if a security flaw is found and make sure you have the ability to
completely restore the hard drive if upgrades don't go well. So I don't
frighten you too much, the Debian package maintainers are amazing, so apt-get usually works very well.
Use
apt-cache to search the local database for
available packages. apt-cache search [search terms] will find packages that
sound like what you want and: apt-cache show [packagename] will return more details
on a particular package. apt-cache showpkg [packagename] will return more
details on a particular package. apt-cache policy [packagename] will return which
versions are available along with the priority of each version. apt-setup will enable you to change mirrors. The
alternative is to edit /etc/apt/sources.list manually (which I prefer). apt-get clean clears the local repository of all retrieved
package files. apt-get autoclean clears the local repository of retrieved
package files of programs that are no longer installed. dpkg -l [packagename] will list the version and a short
description of the package we have installed.
You can also search for packages at http://www.debian.org/distrib/packages. At a later time you can study these great instructions for searching your local package database: http://newbiedoc.sourceforge.net/tutorials/apt-get-intro/info.html.en Also grab http://www.oreilly.com/catalog/linuxnut4/chapter/ch05.pdf for later review.
We need to add 'unstable' and 'testing' sources to our list of available Debian packages.
This modification is critical to the way our system functions.
cp /etc/apt/sources.list /etc/apt/sources.backup
This creates a backup file. Then: vi /etc/apt/sources.list |
At this point, the contents of the file look something like this:
#deb file:///cdrom/ sarge main deb http://mirrors.kernel.org/debian/ stable main deb-src http://mirrors.kernel.org/debian/ stable main deb http://security.debian.org/ stable/updates main We need to modify this file so the result will look something like this: (with only the http server unique to your particular system) deb http://mirrors.kernel.org/debian/ stable main non-free contrib deb-src http://mirrors.kernel.org/debian/ stable main deb http://security.debian.org/ stable/updates main deb http://mirrors.kernel.org/debian/ testing main non-free contrib deb-src http://mirrors.kernel.org/debian/ testing main deb http://mirrors.kernel.org/debian/ unstable main non-free contrib deb-src http://mirrors.kernel.org/debian/ unstable main deb http://www.backports.org/debian/ sarge-backports main contrib non-free Note what I have done here: The line #deb file:///cdrom/ sarge main has been erased. ([up-arrow] to the top of the file and hold down the [Delete] key.) The 4 'testing' and 'unstable' lines have been copied from the top 2 lines, and then modified slightly as indicated. The words "non-free" and "contrib" have been added to 3 of the lines. The backports.org source was added. You are welcome to simply copy and paste what I have listed above. Save and exit the file. |
Enter "i" to get into write mode, highlight the 2 lines with your mouse then right click your mouse in the PuTTY window.
I suggest increasing the apt cache limit.
echo 'APT::Cache-Limit "25165824";' >> /etc/apt/apt.conf
|
You must run apt-get update next.
apt-get update
|
We need to remove some programs for security reasons. Hopefully you don't need PCMCIA or printer support. This server will not need dial-up support either. You will not necessarily have all of these programs installed.
apt-get remove ipchains lpr nfs-common portmap pidentd pcmcia-cs pppoe pppoeconf ppp pppconfig
|
******************************************************
Prevent libc6 upgrade problems:
Home******************************************************
The Linux kernel is in the middle of a transition. If you install Debian using the Sarge CD, you may experience a couple of issues if you were to install any program from 'testing' or 'unstable' that also installs the newest (testing or unstable) version of libc6 as a dependency. This assumes upgrading libc6 will also upgrade the kernel, which has been true in the past but may or may not be true at the moment. If you used an Etch CD to install Debian, these instructions do not apply to you. Newer versions of Debian (Etch for example) use a newer Linux kernel which uses udev instead of devfs for management of devices. These device managers work in different ways from each other, and only one can be used. Part of the kernel is in the form of an initialization ramdisk image. The older kernels use programs from 'initrd-tools' to create the ramdisk, and newer kernels make use of programs included in 'initramfs-tools' or 'yaird' (Yet Another mkInitRD). Although its use is deprecated, it appears initrd-tools can still be used with newer kernels to build an initrd.img provided a version more recent than the one supplied with Sarge is used (but I admit I'm not absolutely certain this statement is accurate). Since the kernel and one or the other of these tools depend on each other, installing an Etch or Sid version of libc6 may try to remove the Sarge 'stable' version of initrd-tools because it is not compatible. Subsequently an attempt to remove the kernel may also be made. Simply removing the kernel is of course not acceptable, so we need to find a way to fix this situation.
There are three different ways to solve this problem. The first (and best) option is to simply never use programs compiled with the new version of libc6 and never upgrade to a new linux-image kernel (the old kernels are called kernel-image). At least not until you are ready to upgrade to Etch that is. The last kernel-image(s) were 2.4.27 and 2.6.8. This means either stay with programs strictly from 'stable' and sarge-backports (and clamav from volatile) and never install programs from 'testing' or 'unstable' that depend on the new version of libc6. Not all programs rely on a newer libc6, so this does not mean you cannot install any programs from these sources, but it does mean you should always test to see if a new program you are installing wants to upgrade libc6 (and remove or upgrade your kernel). To test, simply first 'simulate' the installation with the '-s' argument, e.g.: 'apt-get -s install [...]'. It's always a good idea to do so anyway. A second option to prevent problems is to install initrd-tools from 'testing' (actually, an older testing version you get from me) just prior to upgrading the kernel. Doing so may prevent a libc6 or kernel upgrade from trying to remove the kernel. This does not necessarily prevent a kernel upgrade and migration to udev when libc6 is upgraded however. The third option is to preemptively upgrade the kernel to one of the newer linux-image kernels. Since the newer kernels use udev, this means your system will go through some somewhat radical changes. In some cases devices that used to work may no longer be detected and it is not impossible that some manual intervention may be needed to get the system to recognize them. If you are using RAID I cannot recommend upgrading from one of the older kernel-image kernels to one of the new linux-image kernels. Doing so may leave you with an unbootable system. Kernel upgrades on a RAID system (other than security updates) are generally not a good idea in my opinion. If you need RAID and you intend on upgrading the kernel in the future, you may be better off starting with an Etch CD, and configuring RAID during the installation.
The three methods above are not mutually exclusive. In other words, you could choose all of them. If you choose to upgrade initrd-tools and then later upgrade to a linux-image kernel, doing so may simply prevent removal of the old kernel. If you do not upgrade initrd-tools then the old kernel may be removed, but since we would be replacing the kernel, this is acceptable. If you desire to install only 'stable' programs, this does not prevent you from upgrading the kernel if you desire, but upgrading the kernel would not be necessary.
|
Do not perform any steps listed here until you read the notes above.
If you have chosen to keep your Sarge system 'stable' and only install programs from 'stable' (and optionally 'sarge-backports' and clamav from 'volatile') then you do not need to perform any of these steps. I also don't recommend doing them unless you plan on upgrading the kernel. I will assume you have a 'testing' source in /etc/apt/sources.list. If you have chosen to install an Etch version of initrd-tools, first see if initrd-tools is installed:
dpkg -l initrd-tools | grep ii
If this program is installed, it should report something like "ii initrd-tools 0.1.81.1 tools to create initrd image for prepackaged". If it is not installed, you may already be running a newer kernel that does not use initrd-tools (test with 'uname -r'). If that is the case, there is no need to continue. Otherwise, upgrade initrd-tools:
cd /usr/local/src
If you don't plan on upgrading the kernel right away then you should probably prevent initrd-tools from automatically upgrading to a newer version during 'apt-get upgrade' because that newer version will upgrade libc6! Put the package on hold:
echo "initrd-tools hold" | dpkg --set-selections
If you are currently running a 2.4 kernel ('uname -r' to find out) then you should run these next commands regardless whether you upgraded initrd-tools or not. This may possibly solve a problem where an initrd-img cannot be created for the 2.4 kernel when libc6 is upgraded:
cd
If you choose to install the new 'testing' linux-image kernel you will need to choose the kernel that is correct for your processor. The 486 image will work with Intel or AMD. Obviously you can no longer use an actual 386 chip: linux-image-486 - Linux kernel image on 486-class machines linux-image-686 - Linux kernel image on PPro/Celeron/PII/PIII/P4 machines linux-image-k7 - Linux kernel image on AMD K7 machines For the complete list of available linux-image kernels:
apt-cache search linux-image | grep linux-image
Always first simulate installations to make sure there are no surprises (edit to suit your CPU if you like): apt-get -s -t testing install linux-image-486
If everything looks OK (it will either remove one kernel and install another, or simply upgrade to the new kernel), perform the install: apt-get -t testing install linux-image-486
You may be asked to (re)Configure Locales. If the existing kernel is going to be removed, you will be asked if you know exactly what you are doing. You are going to have to answer 'Yes' (Yes - the whole word) if you want to replace the kernel. Either you will be OK, or your system will be hosed. If it previously did not say that a new kernel will be installed, then it should be obvious that 'Yes' is the wrong answer. Accept the default 'Y' to remaining prompts. Once the new kernel is installed, cross your fingers and reboot: reboot
Once the system comes back up - what kernel are we using? uname -r
This should show you are running kernel 2.6.18 or higher. Since we have upgraded libc6-dev to the 'testing' version, you will also need to upgrade gcc-3.3 to the 'testing' (etch) version:
apt-get -t testing install gcc-3.3
|
If you are using a multi-processor machine, then use a multi-processor kernel!
If your kernel version is 2.4.x, or 2.8.6 or earlier, you will want
to upgrade to an smp version of a 'kernal-image' kernel. Use 'uname -r' to
determine the current kernel, then to find similar available smp kernels, run:apt-cache search kernel-image | grep smp | grep kernel-image
For example, if you were running a 2.6.8 (Sarge) kernel, and had a dual core Intel system, you could use the 'kernel-image-2.6-686-smp' kernel. You would pick the kernel that most closely matches your system (and your current kernel). To install it, you would simply run: apt-get install kernel-image-2.6-686-smp If you were running a newer kernel (2.6.12 or higher - Etch) you would look at the available 'linux-image' smp kernels: apt-cache search linux-image | grep smp | grep linux-image
If you were to install a new kernel, please reboot afterwards.
|
******************************************************
Navigating the system:
Index******************************************************
A quick word about
less. less is a great file
and directory viewer.
You can [page-up] and [page-down] and search for text in the same manner you search for text using vi.
Use a lower case "q" to exit
less.
To view a file using
less: less /path/file
To view directory contents, pipe it through
less:
ls -l | less (current directory, or)
ls -l /path/directory | less
I also like this one:
history | less
then enter an upper case G to go to the bottom of the display.
It displays the commands you have entered previously. Use q to quit. You can even search the same way you do with vi.
I don't mean to break your concentration, but there is another cool tool called
locate.
locate allows you to search a database of every file name on the
system.
It's kind of like Windows Find. You first have to build the database with the
updatedb command, and then you can search through it.
Try this: we are going to use
locate and less
together:
updatedb
locate kmap | less
What you are looking at is every keymap file on the system along with any file name that has the string "kmap" in it.
Play with it; then "q" to exit
less
Now we are going to do something cool.
Take your mouse and highlight any directory you see above, only highlight the directory and not past it.
For example:
/usr/share/keymaps/i386/qwerty/
Now right click your mouse anywhere on the screen. You will notice the text has been pasted to the command line.
Now [left-arrow] over to the beginning of the line (or hit [Ctrl]+a) and type in:
cd
Put a space after
cd and hit [return]. We just saved ourselves
having to type the entire path name just in order to change to that directory.
I like that.
OK, simply enter
cd to get back home.
Sorry for the diversion.
******************************************************
Create Firewall Rules:
Index******************************************************
We need to set up some form of firewall on this box. This is a subject that could (and does) fill volumes. We are going to use something quick and simple that will give us a basic firewall. Something is MUCH better than nothing, and we just don't have time to read volumes on the subject right now.
I worked for a couple days trying to figure out what iptables was all about. I tried using tools like lokkit and shorewall and others, only to get frustrated and confused because I kept getting errors and the firewall simply would not work. My best guess is iptables did not like any rules file it did not create itself. Lokkit was a snap in Red Hat, and a nightmare in Debian. That's pretty much how this whole experience went. But I'm learning a LOT more about GNU/Linux by working with Debian. After day 2 it dawned on me iptables is somewhat like Cisco access lists (which I am a little familiar with).
I am going to give you a set of commands below that I want you to paste into the command line, in the correct order. You MUST change the IP addresses to fit your needs, if you have not already done so. The line with '--dport 22' on it is SSH and the network address to the left needs to be the network that both your computer and the spamfilter computer are on. You could also limit access to a single computer (yours, of course) by using your_ipaddress_goes_here/32. This is a security measure. If you do that part wrong, it will lock you out. The lines with '--sport 53' on them are for access to DNS servers. BTW, all you have to do to change your DNS servers is change the entries in /etc/resolv.conf.
If you would like to add more rules in the future or make modifications, simply copy and paste these lines into a text editor like notepad, make the changes you would like, and then copy and paste them to a command prompt in your PuTTY window. You can copy and paste all the lines at once. The first line deletes all the entries that were in the rule-set previously and the next to the last line saves the new rule set. The last line shows how one would load a rules file into iptables. Keep a copy of the text file on your computer and call it firewall-rules.txt. I learned to never edit the /etc/firewall-rules file directly on the spamfilter computer. It looks like iptables will reject the file if anyone other than itself has modified it.
DO NOT USE AS IS, CHANGE NETWORK ADDRESS FIRST IF YOU HAVE NOT ALREADY DONE SO:
You can copy and paste this whole section to the command prompt:
iptables -F
|
Now run:
iptables -L |
We have written the firewall rules to a file on the spamfilter computer, but iptables starts with an empty rule set each time the computer restarts. The rule set we saved to /etc/firewall-rules must be "loaded" into iptables every time the system starts up.
We are going to insert the command to configure iptables into a file that starts up the network interfaces when the system boots up:
vi /etc/network/interfacesAnd insert the following text (remember, it's "i" to insert) in the blank line just below "iface lo inet loopback": pre-up iptables-restore < /etc/firewall-rules
Save and exit the file as usual with [Esc] : wq From now on I will assume you know how to edit, save, and exit files using vi. If not stated, it will be implied that after editing a file, you need to save and exit it, or if necessary, discard changes and start over. |
This is not the place, and not the way, to do so. That's a whole 'nuther subject. This file is the right place (along with /etc/resolv.conf) to change network settings however.
That's all there is to it. You have just used what I believe is the fewest possible steps to create a simple functional personal firewall for this machine. I will admit that it should have been a lot easier by utilizing one of the firewall tools, but it just didn't work out.
At this point our firewall allows external users to connect to SSH and Mail. It also allows replies from Pyzor, DCC, DNS servers and NTP servers. It blocks (I hope) everything else except any sessions that originate from us. This allows us to connect to the outside world. This box should be behind another firewall at any rate. If so, that firewall/screening router will need to be configured to allow tcp port 25 traffic to this machine, but only after this box is fully functional. If you have things locked down really tight; take a look at http://flakshack.com/anti-spam/wiki/index.php?page=Provide+firewall+access for some ideas. Keep in mind we also need udp port 24441 for Pyzor and access to external DNS servers. As far as DCC, Razor and Pyzor go, try them before you start messing with your Internet firewall. I have my spamfilter behind a screening router, a hardware firewall, and software NAT box firewall and none of them required reconfiguration for these programs to work. Port 25 SMTP will probably need to be opened however.
If you have not done so, reboot again and runiptables -L to verify the firewall loaded during start up. |
If you have problems, enter the command
iptables -F from the
console to clear out iptables. This will allow you another shot at it.
******************************************************
Disable Unnecessary Daemons:
Index******************************************************
We are now going to remove some services (daemons) that start up at boot time. I only want you to remove the services I have listed below, no more than that. You could (and probably would) end up with an unusable system if you disabled more than this. Our basic system does not start up many services anyway but "you can't hack a service that isn't running". The only secure system is a system that doesn't exist.
Below is a list of commands I found useful to determine what services were running.
Run them one at a time if you care to.
top
ps afx
ps afxl
ps -A
ls -F /etc/rc2.d
lsof -i | grep LISTEN
netstat -pn -l -A inet
netstat -pn -l inet
These are from http://linuxgazette.net/issue89/gonzales.html#4
I also liked the lsconfig script I found here:
http://www.shallowsky.com/software/scripts/lsconfig
Save it as /usr/bin/lsconfig and make it executable.
Like this:
cd /usr/bin
wget http://verchick.com/mecham/public_html/debian/lsconfig
chmod +x /usr/bin/lsconfig
lsconfig
If you run lsconfig, the stuff just scrolls by on the screen. You can choose "Copy all to Clipboard" from the drop down menu of the PuTTy window. Click on the two little computers in the upper left-hand corner of the PuTTY window to access the menu. Then open a spreadsheet and paste it into it. Play with it from there.
We need to make a backup of the init scripts in /etc/init.d because after we remove some services, the system may delete the scripts.
cp -r /etc/init.d /etc/init.d-original |
These commands assume you are not hooking up a printer to this machine and you are not using a laptop to build your spamfilter!
Feel free to copy and paste these next two boxes in their entirety.
Ok, let's just get this done:
/etc/init.d/lpd stop
|
The inetd service (InterNET Daemon) starts multiple services that can be enabled or disabled individually.
update-inetd --disable time
|
Check that we got everything:
lsof -i | grep LISTEN
The only daemon you should see is at this point is *:ssh You may have to run this again: update-inetd --disable discardIf there are other programs shown, try rebooting and test again. |
If you would like to get any of these services back, we can reverse the events.
For example, to enable 'ident' and then refresh inetd, you could issue these 2 commands:
update-inetd --enable ident
/etc/init.d/inetd restart
For example, to re-enable the printer service, and start it up right now:
update-rc.d lpd defaults
/etc/init.d/lpd start
If you get an error that the file does not exist, first restore it from the backup we made, and then try again:
cp -i /etc/init.d-original/lpd /etc/init.d
update-rc.d lpd defaults
/etc/init.d/lpd start
This is an example only, we actually uninstalled printer support.
We have just made our machine much more secure than when we started. I will talk about additional security measures at the end of this document, but we want to get this thing up and running first! We want to see this puppy actually do something! You used a few commands above to help you see what services were running before we made changes. Run them again, if you like, to see the effect of disabling them.
******************************************************
Configure the NTP daemon:
Index******************************************************
Install the program:
apt-get install ntp-simple
|
/etc/ntp.conf and
insert it per the example in the file. NTP is a flexible and complex system so I leave
it up to you to research it further if you care to.
If you care to choose your own servers from the list of Public NTP Secondary
(stratum 2) Time Servers at
http://www.eecis.udel.edu/~mills/ntp/clock2a.html we can use the little ntpdate
program to quickly test them prior to insertion in /etc/ntp.conf:
For example:
/etc/init.d/ntp-server stopntpdate clock.fmt.he.netntpdate ntp1.tummy.com/etc/init.d/ntp-server start
By the way, the command to modify the date and time is
date
and to change the time zone it's tzconfig
******************************************************
Installing Programs:
Index******************************************************
One of the Perl modules amavisd-new needs is Compress::Zlib (version 1.35 or newer). The Debian 'stable' version of this module (libcompress-zlib-perl) is at version 1.34. If we install the 'testing' version of this program on a system running the 'stable' release, the program is configured to install a new version of libc6 and other critical system files as noted earlier. If you are running 'stable', I DO NOT want this to happen so we will instead install libcompress-zlib-perl from backports.org:
apt-get -t sarge-backports install libcompress-zlib-perl
|
We need to install a number of additional programs. Go ahead and select ALL the text in the box below with your mouse, then use [Ctrl]+c to copy it to the clipboard, then right-click the PuTTY window, then hit [enter] to issue the command. I suggest you select from right to left (bottom to top). Going the other way always wants to select one extra space character which can be a problem with certain apt-get commands.
apt-get install arc arj autoconf automake1.7 bzip2 cabextract db4.3-util libarchive-tar-perl libarchive-zip-perl libauthen-sasl-perl libberkeleydb-perl libconvert-binhex-perl libconvert-tnef-perl libconvert-uulib-perl libdb4.3-dev libdbd-mysql-perl libdbi-perl libdigest-hmac-perl libdigest-sha1-perl libhtml-format-perl libhtml-parser-perl libhtml-tagset-perl libhtml-tree-perl libio-multiplex-perl libio-socket-ssl-perl libio-string-perl unrar unarj libdigest-nilsimsa-perl libmail-audit-perl
|
And there are more:
apt-get install lha libio-stringy-perl libio-zlib-perl libldap2 libmail-spf-query-perl libmailtools-perl libmime-perl libnet-dns-perl libnet-ldap-perl libnet-perl libnet-ph-perl libnet-server-perl libnet-snpp-perl libnet-telnet-perl libsocket6-perl libtimedate-perl libtool libunix-syslog-perl liburi-perl libwww-perl lsof lynx lzop make ncftp nomarch pax perl-doc rblcheck unzip unzoo zip zlib1g-dev zoo pyzor razor
|
For lha license information see http://lists.debian.org/debian-devel/1999/11/msg00549.html and for rar (unrar) see http://www.rarsoft.com/index.htm
This is a new system, so let's make sure everything is current:
First, run a simulation:apt-get -s upgrade
Then if everything is as you might expect: apt-get upgrade
|
If the kernel is upgraded, once the upgrade process is complete, you must:
reboot
|
We are using apt-get to download and install most of our core programs. Note that when we install Postfix, apt-get is smart enough to remove exim4 (because it conflicts with Postfix). This document is based of the following versions of these programs: Postfix 2.1.x spamassassin 3.0.3 and amavisd-new 2.4.5. However, I suggest installing newer versions of Postfix and SpamAssassin. The instructions may differ somewhat if newer (or older) versions are installed. Please run:
apt-cache policy postfix spamassassin
|
This will give output similar to the following:
postfix:
Installed: (none)
Candidate: 2.1.5-9
Version Table:
2.3.2-1 0
400 http://mirrors.kernel.org unstable/main Packages
2.2.10-2 0
450 http://mirrors.kernel.org testing/main Packages
2.2.10-1bpo2 0
350 http://www.backports.org sarge-backports/main Packages
2.1.5-9 0
600 http://ftp.us.debian.org stable/main Packages
600 http://mirrors.kernel.org stable/main Packages
spamassassin:
Installed: (none)
Candidate: 3.0.3-2sarge1
Version Table:
3.1.3-1 0
400 http://mirrors.kernel.org unstable/main Packages
3.1.3-0bpo1 0
350 http://www.backports.org sarge-backports/main Packages
3.1.1-1 0
450 http://mirrors.kernel.org testing/main Packages
3.0.3-2sarge1 0
600 http://security.debian.org stable/updates/main Packages
3.0.3-2 0
600 http://ftp.us.debian.org stable/main Packages
600 http://mirrors.kernel.org stable/main Packages
This tells us there are newer versions of Postfix available in the 'testing',
'unstable' and sarge-backports branches.
If you upgraded the kernel to 2.6.15 or newer, you can optionally install the
newer 'testing' version
by simply installing it using "apt-get -t testing install [list of packages]".
If you are trying to keep the system as 'stable' as possible, but would like to
install something newer than what is provided in 'stable' you would want to install
the version from backports-org "apt-get -t sarge-backports install [list of packages]".
When mixing testing with stable, there is always some potential of
future problems with program dependencies, but generally, Debian is very smart about
these issues. The more cautious person would not use 'testing' or 'unstable'. As mention previously,
installing programs from 'testing' or 'unstable' that also upgrade to the newest version
of libc6 may remove your kernel!
The report shows there is a newer version of SpamAssassin available. We can
upgrade SpamAssassin to the new version once our install is done, or you can
install the newest 3.1 version now from backports.org.
There is a Debian version of amavisd-new available, but we are NOT
going to install it. The configuration files
for newer Debian (testing/unstable) versions of amavisd-new are
not consistent with the typical way amavisd-new is configured. Instead of one
configuration file, the newer Debian versions split the configuration files into half
a dozen files in a couple different directories.
We will instead install amavisd-new from the original author.
So with this in mind:
Read instructions above before you proceed.
apt-get install spamassassin
If you would like to install the backports.org version of SpamAssassin, you should do it like so: apt-get -t sarge-backports install spamassassin |
Read instructions above to determine whether to install the 'stable' version or not.
Also read this.
apt-get install postfix postfix-pcre postfix-mysql postfix-ldapIf you would like to install the backports.org version of Postfix, you would do it like so: apt-get -t sarge-backports install postfix postfix-pcre postfix-mysql postfix-ldap |
|
Debconf will pop up a Postfix configuration screen. For [General type of configuration?] select: No configuration |
If (and only if) you already have amavisd-new installed, you need to remove it
(this will not remove your configuration files which is a good thing). First make
sure amavisd-new is the only thing that will be removed by 'simulating' the removal:apt-get -s remove amavisd-new
If it is, then remove it: apt-get remove amavisd-new
If it is not, then you must make a note of any and all programs that will be removed, because you will have to reinstall them. Good luck with all that. ;) |
Here are all the steps needed to install amavisd-new. Some of the files it tries to delete or copy may not be on your system but there is no need to panic if it fails to find them. The last few dpkg-statoverride commands will also fail if you have ever installed Debian amavisd-new - but this is not an issue either. We are going to place files in the same place the Debian version of amavisd-new would have.
adduser --group --system --home /var/lib/amavis --shell /bin/sh amavis
|
******************************************************
Postfix Configuration Part 1:
Index******************************************************
We need some sample files from the Postfix source code, note that the source code we grab here is the original source code, before it is modified by the Debian package maintainers:
Position yourself:cd /usr/local/srcIf you installed Postfix version 2.1.5, follow these two instructions: wget http://ftp.debian.org/debian/pool/main/p/postfix/postfix_2.1.5.orig.tar.gztar xzvf postfix_2.1.5.orig.tar.gz
If you installed a newer version of Postfix, do this instead: apt-get source postfix |
List the contents of this directory:
ls -l |
When we unpacked the source code, we created a new directory called postfix-2.1.5. Make sure you are still in the /usr/local/src directory, then remove the file we downloaded with the command:
rm postfix* |
We now have a number of Postfix sample files. We want to make use of those sample files so we will copy them to the /etc/postfix directory.
| The second line below will need to be edited if you have a different version of
the Postfix source code. MAKE SURE you answer "n" to "overwrite?" Do each section separately. cp -i /usr/share/postfix/main.cf.debian /etc/postfix/main.cf
cp -i /usr/local/src/postfix-2.1.5/conf/* /etc/postfix
cp -i /etc/postfix/header_checks /etc/postfix/body_checks
cp -i /etc/postfix/access /etc/postfix/sender_access
|
******************************************************
Edit master.cf:
Index******************************************************
Read this before you complete this section.
If you are using Postfix version 2.1.x, 2.2.x or 2.3.x, I have done the work of configuring master.cf for you and you may simply download the file from me. If you wish to use this file, follow the first 4 steps below.
The master.cf we download here can be used with Postfix versions 2.3.x, 2.2.x and 2.1.x postfix stop
cd /etc/postfix
mv master.cf master.cf-original
wget http://verchick.com/mecham/public_html/debian-postfix-2.3-amavisd/master.cf
|
Now you can simply jump to Edit Main.cf:
If you wish to do the work yourself, continue on.
postfix stop vi /etc/postfix/master.cf
|
Next, we want to give Postfix some information it will need to talk to the amavisd-new program. (BTW, watch out for the two Postfix configuration files, both located in the /etc/postfix folder. More than one admin has gotten confused between "master.cf" and "main.cf"!)
| Add these lines near the bottom of master.cf. Note: the items on these lines are separated by tabs. And the "-o" is the lower case letter o, not zero. These settings are from http://www.ijs.si/software/amavisd/README.postfix. You can copy and paste this entire section once the cursor is in the correct position (see below) and you are in insert mode. Note: when copying sections like this that contain tabs, rather than using a right click of the mouse to paste into the editor, press [Shift]+[Insert]: |
smtp-amavis unix - - - - 2 smtp
-o smtp_data_done_timeout=1200
-o smtp_send_xforward_command=yes
-o disable_dns_lookups=yes
-o max_use=20
127.0.0.1:10025 inet n - - - - smtpd
-o content_filter=
-o smtpd_restriction_classes=
-o smtpd_delay_reject=no
-o smtpd_client_restrictions=permit_mynetworks,reject
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o smtpd_data_restrictions=reject_unauth_pipelining
-o smtpd_end_of_data_restrictions=
-o mynetworks=127.0.0.0/8
-o smtpd_error_sleep_time=0
-o smtpd_soft_error_limit=1001
-o smtpd_hard_error_limit=1000
-o smtpd_client_connection_count_limit=0
-o smtpd_client_connection_rate_limit=0
-o smtpd_milters=
-o local_header_rewrite_clients=
-o local_recipient_maps=
-o relay_recipient_maps=
-o receive_override_options=no_header_body_checks,no_unknown_recipient_checks
|
We also need to add two items below the 'pickup' service type. The 'pickup' service 'picks up' local mail (local meaning "on this machine") and delivers it. Later we will create a daily/weekly report that this box will mail to us and because the report will contain contents that will classify the report itself as spam, this is a way to bypass content filtering for mail generated by this machine.
Add this just below the 'pickup' service type:
-o content_filter=
-o receive_override_options=no_header_body_checks
|
When you are all done, the table and the lines right after it should end up looking like this:
# ==========================================================================
# service type private unpriv chroot wakeup maxproc command + args
# (yes) (yes) (yes) (never) (100)
# ==========================================================================
smtp inet n - - - - smtpd
# -o receive_override_options=no_address_mappings
#submission inet n - - - - smtpd
# -o smtpd_enforce_tls=yes
# -o smtpd_sasl_auth_enable=yes
# -o smtpd_client_restrictions=permit_sasl_authenticated,reject
#smtps inet n - - - - smtpd
# -o smtpd_tls_wrappermode=yes
# -o smtpd_sasl_auth_enable=yes
# -o smtpd_client_restrictions=permit_sasl_authenticated,reject
#628 inet n - - - - qmqpd
pickup fifo n - - 60 1 pickup
-o content_filter=
-o receive_override_options=no_header_body_checks
cleanup unix n - - - 0 cleanup
qmgr fifo n - n 300 1 qmgr
#qmgr fifo n - - 300 1 oqmgr
tlsmgr unix - - - 1000? 1 tlsmgr
rewrite unix - - - - - trivial-rewrite
bounce unix - - - - 0 bounce
defer unix - - - - 0 bounce
trace unix - - - - 0 bounce
verify unix - - - - 1 verify
flush unix n - - 1000? 0 flush
proxymap unix - - n - - proxymap
smtp unix - - - - - smtp
# When relaying mail as backup MX, disable fallback_relay to avoid MX loops
relay unix - - - - - smtp
-o fallback_relay=
# -o smtp_helo_timeout=5 -o smtp_connect_timeout=5
showq unix n - - - - showq
error unix - - - - - error
discard unix - - - - - discard
local unix - n n - - local
virtual unix - n n - - virtual
lmtp unix - - - - - lmtp
anvil unix - - - - 1 anvil
scache unix - - - - 1 scache
#
smtp-amavis unix - - - - 2 smtp
-o smtp_data_done_timeout=1200
-o smtp_send_xforward_command=yes
-o disable_dns_lookups=yes
-o max_use=20
127.0.0.1:10025 inet n - - - - smtpd
-o content_filter=
-o smtpd_restriction_classes=
-o smtpd_delay_reject=no
-o smtpd_client_restrictions=permit_mynetworks,reject
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o smtpd_data_restrictions=reject_unauth_pipelining
-o smtpd_end_of_data_restrictions=
-o mynetworks=127.0.0.0/8
-o smtpd_error_sleep_time=0
-o smtpd_soft_error_limit=1001
-o smtpd_hard_error_limit=1000
-o smtpd_client_connection_count_limit=0
-o smtpd_client_connection_rate_limit=0
-o smtpd_milters=
-o local_header_rewrite_clients=
-o local_recipient_maps=
-o relay_recipient_maps=
-o receive_override_options=no_header_body_checks,no_unknown_recipient_checks
|
(Don't worry about the stuff below this part displayed above - you won't need to change any of those rows, and they are all listed as "pipe" in the last column.)
******************************************************
Edit main.cf:
Index******************************************************
Our next friend is the file
/etc/postfix/main.cf the main
configuration file for Postfix. Following are suggested values to use in
main.cf. These have been tested for this configuration and will work fine, but
there are many judgment calls involved in this, and it is a good idea at some
point to learn more about Postfix configuration, on your own. You could first
look at the sample Postfix main.cf file /usr/share/postfix/main.cf.dist
. There are comments describing some of the most common options.
Refer also to the Postfix documents on your machine in the
/usr/local/src/postfix-2.1.5/README_FILES directory, or
read the documentation on the Postfix web site
http://www.postfix.org/documentation.html. I also recommend
http://www.postfix-book.com/.
Since we are setting up our spamfilter to relay all of its mail to another server, we will be using what Postfix considers a "relay domain address class" which essentially means that we will use, out of the 300+ configurable parameters in Postfix, a small group of parameters that serves our purpose best. This address class is described here: http://www.postfix.org/ADDRESS_CLASS_README.html#relay_domain_class. We are also acting as a primary MX for another server so please read this appropriate section: http://www.postfix.org/STANDARD_CONFIGURATION_README.html#backup
It is common in Postfix to store items in lookup tables. We are going to use several hash tables to store data that Postfix will use. Once we have plain text data in these tables, we use the
postmap
command to create binary files (Berkeley DB format) that Postfix will ultimately use to
retrieve the data. For example, if you have a file called "filename" and you "postmap filename",
a new file is created "filename.db". Postfix will retrieve data from "filename.db", not
"filename".
There are more than a dozen other types of data files that Postfix
can use to store data. Hash tables are an appropriate choice for
several tables we will use, and pcre (Perl Compatible Regular Expressions)
is appropriate for a couple tables we will use to hold content filtering data.
In its simplest form a hash table is comprised of 2 pieces of data,
a key and a value; typically referred to as the key/value pair. The key
and the value are separated with whitespace (typically a space or tab).
The data in a typical table that we use in Postfix would look something like:user1@example.com OK
user2@example.com OK
user1@example2.com OK
Suggested reading: http://www.postfix.org/DATABASE_README.html
OK, lets get going. Note: in commands, wherever quote marks " " are used, use them. Rather than editing main.cf directly (which you may nonetheless do, if you prefer) we'll use a handy tool that comes with Postfix, named "postconf". We will use the -e switch, which means to "edit" main.cf.
We simply need to make a correction to the default setting here:
postconf -e "alias_maps = hash:/etc/aliases"
|
Now we will create from the text version of the aliases file, the binary version that Postfix will actually use. We do not need to edit the aliases file at this time but it would be a good idea to do so simply to view the contents. You need to run
newaliases now, and every time after you edit the aliases file. The
newaliases command is just like postmap except that it's
specific to the aliases file.
newaliases
|
myorigin The domain name that mail created on this machine appears to come from. For example, if one of our programs (cron for example) sends mail to "root" it will be sent to "root@example.com".
postconf -e "myorigin = example.com" |
myhostname The fully-qualified domain name (FQDN) of the machine running the Postfix system.
postconf -e "myhostname = sfa.example.com" |
mynetworks These are the machines I trust, and will relay mail for, to any destination. Generally, this is set to my LAN, or just one, or a few trusted internal mail servers. Along with "relay_domains", this is an important one to get right lest you run the possibility of becoming an "open relay". In other words, your box could accept and forward mail to domains for which it has no business doing so. Being an open relay is a serious issue, and can cause you to get blacklisted by various Internet anti-spam lists, among other problems. You can specify a single computer, multiple individual computers, or any computer on a specified network. You can also exclude certain hosts in your network by preceding the IP address with an exclamation point. If you are using a NAT router that substitutes the real client IP address with its own, then you must exclude the IP address of the NAT router from mynetworks. If you will be dealing with multiple internal mail servers, and/or want to allow several machines and/or subnets to relay through this server (careful!), just add them to this parameter in CIDR format, like this:
Please read important notes above.postconf -e "mynetworks = 127.0.0.0/8, 222.222.222.222/24, 10.10.10.10/24"
|
message_size_limit Maximum size email that Postfix will let in the "front door".
postconf -e "message_size_limit = 10485760" |
local_transport Give an error message for local delivery attempts.
postconf -e "local_transport = error:no local mail delivery" |
mydestination An empty mydestination tells Postfix this machine is not the final destination.
postconf -e "mydestination = " |
local_recipient_maps An empty local_recipient_maps tells Postfix there are no local mailboxes.
postconf -e "local_recipient_maps = " |
virtual_alias_maps Our spamfilter must be able to receive mail for postmaster@[111.111.111.111]. Reportedly, some things actually expect this ability to exist. We will also allow mail to abuse@[111.111.111.111]. Since we do not allow local mail delivery, mail addressed to our spamfilter IP address will get rejected with an error message. Setting up virtual_alias_maps allows email to these two accounts to be forwarded to an inside address. Make sure your Exchange server is set up to receive messages addressed to "root", "postmaster" and "abuse".
Set up a reference to the virtual file:postconf -e "virtual_alias_maps = hash:/etc/postfix/virtual"Then edit the virtual file: vi /etc/postfix/virtualand add postmaster and admin in the format:
postmaster postmaster@example.com
Save and exit the file, then create the binary file that Postfix will use:postmap /etc/postfix/virtual |
relayhost Generally speaking, if this machine is on an internal network (with no public IP address), you may need to configure 'relayhost'. If it is on the Internet serving as a gateway server, you would only configure 'relayhost' if you must relay your mail through some other server (like your ISP's), otherwise you would not. The relayhost is the email server you wish to use to send outbound email. The outbound email I'm talking about is the non-local mail, mail bound for domains other than ours, not the normal email that comes from outside the system and is bound for your internal mailboxes (we use /etc/postfix/transport to route those). At this time you are using a machine other than this one to send mail out to the Internet so you could place the IP address of that machine here (the brackets must be there). If preferred, you can use a host name instead of an IP address (keep the brackets, i.e. [gateway.example.com]). If this is blank, or not configured, then Postfix will use DNS to deliver outbound mail (which is the ideal configuration). It is best to not use relayhost, but if you do not, you should first have your reverse DNS record in place and of course your "A" record and "MX" record so other servers on the Internet will accept mail from this machine at this IP address. If these are not yet in place, it may be useful to temporarily configure relayhost. If you point relayhost to the Exchange server, the Exchange server (or other SMTP server) must be configured to allow our spamfilter to relay mail through it http://www.msexchange.org/pages/article.asp?id=54. You need this so bounces have a way out of the network.
Read the notes above before you enter this: postconf -e "relayhost = [666.666.666.666]" |
relay_recipient_maps - We are going to build a table of every single user in every single domain that we accept mail for. This table will be used to reject mail that is addressed to nonexistent users in our domain(s). Don't freak out just yet. At this time we are only going to set up the structure of the table. Then you will come back to this after your spamfilter is functional and work on finding an automated process that can be used to populate the database. Or, if you have a manageable number of users, manually enter them. If this proves too daunting a task, we may have to rely on the Exchange server to do the 'user unknown' rejects. In some cases you might be able to use reject_unverified_recipient as an alternative. If you are in fact using Exchange, there are HOWTOs available that describe automating the process of building the relay_recipients table. It has been very common of late for spammers to launch 'dictionary attacks'; sending thousands of messages to a domain using fabricated user names. Our spamfilter will have to process each and every one of these unless you put your valid users in the relay_recipients table. Don't underestimate the importance of this and make sure you are not the only one in your organization who knows how to make changes to this file.
Set up a reference to a file we will create to store the data:postconf -e "relay_recipient_maps = hash:/etc/postfix/relay_recipients"
Then edit that file: vi /etc/postfix/relay_recipients
For the moment, we are going to accept mail for all users in our domain(s) so enter each domain you accept mail for in the format:
@example.com OK
Then create the binary file that Postfix will use: postmap /etc/postfix/relay_recipientsThe entries above are temporary. They are wildcards that allow mail to your domains. You MUST remove the entries above at some point in the near future and replace them with every single one of your valid recipients' email addresses. When you are ready to enter each user individually in the relay_recipients file, you would first remove (or comment out) the data above that allows mail to all users in the domain, and then list each user individually in the form: user1@example.com OK user2@example.com OK user3@example.com OK |
http://www2.origogeneris.com:4000/relay_recipients.html - http://www-personal.umich.edu/~malth/gaptuning/postfix/ - http://www.unixwiz.net/techtips/postfix-exchange-users.html - http://postfix.state-of-mind.de/patrick.koetter/mailrelay/
transport_maps Tells Postfix where to look for a transport file. We use the transport file to tell Postfix where to forward valid mail addressed to our domain(s). Our file will be
/etc/postfix/transport and we
will set it up in similar fashion to relay_recipients.
Create a reference to it in main.cf: postconf -e "transport_maps = hash:/etc/postfix/transport"
Then edit it: vi /etc/postfix/transport
Add 1 new line for each domain for which you will be handling mail, similar to the example below. The IP address is that of whatever server is the final destination of messages addressed to our domain(s) (our Exchange server). It does not matter where you place these items in the file, but I like to put them at the top. example.com smtp:[666.666.666.666]
(DO include the brackets on these lines!) You can also use a FQDN hostname instead of an IP address (i.e. smtp:[exchange.example.com]). Then create the binary file Postfix will use: postmap /etc/postfix/transport |
relay_domains What destination domains (and subdomains thereof) this system will relay mail for. You want to list here ONLY domains for which you are responsible for accepting mail. In addition to allowing mail to be relayed to these domains, this setting also infers that we do not relay mail to domains not listed here and therefore this is a critical anti-relay control setting. Separate the list with commas (technically, spaces also work).
postconf -e "relay_domains = example.com, example2.com, example3.com"
|
example.com 1
example2.com 1
(or)
example2.com OK
then set "relay_domains = hash:/etc/postfix/relay_domains" in main.cf, then "postmap /etc/postfix/relay_domains" and "postfix reload" at the command prompt. See http://www.postfix.org/postconf.5.html#relay_domains for details.
recipient_delimiter If your current SMTP/POP3/IMAP server is configured to use address extensions (for example user+foo@example.com) then
recipient_delimiter
should be set to match the delimiter you are currently using to
separate the user name from the address extension. Amavisd-new
also uses $recipient_delimiter and this setting needs to match the
Postfix setting. This has nothing to do with the comma you are using to separate
multiple people you send email to (user1@example.com, user2@example.com, user3@example.com)
using your email client (MUA).
|
Typical settings - Choose one I Don't use recipient delimiters: postconf -e "recipient_delimiter = "
I currently use the plus sign: postconf -e "recipient_delimiter = +"
I currently use the minus sign: postconf -e "recipient_delimiter = -"
|
If (and only if) the IP address you present to the world is not the IP
address of your spamfilter (you are configured to run behind a NAT firewall
or a proxy server) please add these two lines to main.cf, then you must
uncomment and configure proxy_interfaces (1.2.3.4 represents the public
IP address):# Specify your NAT/proxy EXTERNAL address here. |
Address rewriting. When using a content_filter like amavisd-new,
because mail is sent to Postfix twice, address rewriting takes place twice unless we
disable it either before amavisd-new or after amavisd-new has processed a message. Our
virtual file is one example of a table that rewrites addresses. It is up to you
to decide whether you want amavisd-new to see the original address, or the rewritten
address. In master.cf you can disable address rewriting before amavisd-new by
setting:smtp inet n - - - - smtpd -o receive_override_options=no_address_mappingsBe careful, there are two lines that look similar. You are looking for smtpd on the right hand side. To instead disable address rewriting when amavisd-new returns (non discarded) messages to Postfix, you would edit the current receive_override_options override on the reinjection port (127.0.0.1:10025). For example, if we now have:
127.0.0.1:10025 inet n - - - - smtpd
[...lots of overrides are here...]
-o receive_override_options=no_header_body_checks,no_unknown_recipient_checks
You would change this to:
127.0.0.1:10025 inet n - - - - smtpd
[...lots of overrides are here...]
-o receive_override_options=no_address_mappings,no_header_body_checks,no_unknown_recipient_checks
vi /etc/postfix/master.cf and edit one or the other.
|
******************************************************
Postfix Anti-Spam settings:
Index******************************************************
Preliminary Notes:
When a client (a computer trying to send us mail) connects to Postfix and begins a communication session, Postfix records information about that session. Prior to the point where Postfix accepts mail from that session for delivery, we have the option of evaluating the session and rejecting the mail by setting some restrictions in main.cf. This link illustrates what happens during a typical SMTP session: http://helpdesk.islandnet.com/pep/smtp.php. The restrictions below help ward off some spam and prevent our system from becoming an open relay. These restrictions cause some mail to be rejected by Postfix right at the "front door". This will save system resources because the mail will not enter our system and therefore will not be scanned by amavisd-new (and subsequently by SpamAssassin). This is good and bad. It saves system resources, but it also doesn't let you see the rejected mail. All you will see is a log entry or two from Postfix saying essentially "Hey, a mail server named "xxxxxxxx" at IP address X.X.X.X tried to send some mail in, but it broke rule XYZ, so I rejected it". Now this could have been spam (spammers often intentionally don't follow RFCs in order to accomplish their goals) but if it was a "legitimate" mail, say from a customer whose IT Department has simply misconfigured their mail server (it happens), you could have some ruffled feathers to deal with.
If you want to allow ALL mail addressed to us to come in the front door, and therefore allow amavisd-new/SpamAssassin to handle all spam control within the system, you would need to modify a couple of the settings below. I personally find a combination of Postfix and SpamAassassin anti-spam control to be best. At the very least you need to insure you do not disable the built in default anti-relay control in Postfix (smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination).
The configuration below is actually very conservative, allowing most email to come in the front door so amavisd-new and SpamAssassin have their shot at it. For me it (safely) rejects about 35% of the email bound for my users, so I think these settings are quite valuable. I recommend this approach to start. Adding additional restrictions will increase the likelihood of rejecting valid email from improperly configured computers. If you decide to add/remove permissions/restrictions in the future, do so one at a time and give yourself ample time to evaluate the effect of the change. I strongly suggest you actually have a good understanding of how these restrictions work before you make changes to the entries below. Among other things, getting this stuff wrong could reject legitimate mail and/or cause us to become an open relay. Note that restrictions don't always restrict, some also permit.
If you want to gain a better understanding of these settings, good resources are http://jimsun.linxnet.com/misc/postfix-anti-UCE.txt, http://www.postfix.org/SMTPD_ACCESS_README.html, http://www.securitysage.com/antispam/, the somewhat dated (2001) http://www.mengwong.com/misc/postfix-uce-guide.txt, and this excellent book. You will note I only use a couple of the same settings as these, so this configuration is no where near as restrictive. Keep in mind that SpamAssassin and amavisd-new will come into the picture in a bit, and will provide us with much more flexible and configurable options to recognize and manipulate spam.
smtpd_helo_required: Make any connecting mail server do a proper smtp "handshake" and announce its name. Internet RFCs require this, so we do too.
postconf -e "smtpd_helo_required = yes" |
Preface: Postfix' restriction stages are as follows, and are processed in the following order:
smtpd_client_restrictions
smtpd_helo_restrictions
smtpd_sender_restrictions
smtpd_recipient_restrictions
smtpd_data_restrictions
We are only going to place entries in the last three restriction stages. Restriction stages are processed in this order regardless of the order listed in main.cf
smtpd_sender_restrictions: This restriction stage restricts what sender addresses this system accepts in MAIL FROM: commands (the envelope sender). We will place three tests (restrictions) in this restriction stage.
A restriction stage holds a list of restrictions (tests). Typically, tests evaluate to either DUNNO, REJECT, or OK. DUNNO means "I don't know what to do, let the next test decide". REJECT simply rejects the mail. OK means no more tests are performed in this restriction stage, tests continue with the next stage (if any). Reject_* type tests typically evaluate to REJECT or DUNNO. Permit_* type tests typically evaluate to OK or DUNNO, and check_*_access type tests can perform a variety of actions. The illustration shows the basic logic.
SMTP session
|
V
restriction stage-------------
test ---------------REJECT->
| \
| DUNNO
| \
| V
| next test------REJECT->
| | \
OK OK DUNNO
| | \
| | V
V V
next restriction stage-------
|
2) reject_non_fqdn_sender Reject when the envelope sender mail address is not in the proper format. Remember, the "envelope sender" is what the sending mail server gives in the "MAIL FROM:" line during the SMTP session, not the header "From:" line. "Joe" is not allowed to send us mail (because we can't reply to "Joe") but "Joe@example.com" is at the very least an email address. If the sender does not get rejected at this point, this test evaluates to "DUNNO".
3) reject_unknown_sender_domain Reject when the envelope sender's domain part of the mail address has no DNS "A" or "MX" record at all. This setting kicks about 35% of the mail coming in my mail server. It is common for spammers to use a bogus domain name so they don't have to deal with the backlash of rejected mail. It is also important for us not to fill up our queue with Bounce notices that can never be delivered due to the fact that the sender's domain does not even exist. If the sender's domain has an "A" or "MX" record, this test will also evaluate to "DUNNO". On occasion, you will see in a report that someone you wish to receive mail from has been rejected by this setting. One possible cause of this is when legitimate senders deliberately use bogus domain names so you will not reply to them. This is where the sender access list comes in handy. You can give them an OK there, and this test will be bypassed.
postconf -e "smtpd_sender_restrictions = check_sender_access hash:/etc/postfix/sender_access,
reject_non_fqdn_sender, reject_unknown_sender_domain" |
smtpd_recipient_restrictions: The access restrictions that the Postfix SMTP server applies in the context of the RCPT TO: command. This refers to the "envelope recipient" which is what the client gave in the "RCPT TO:" line during the SMTP session, not the header "To:" line. smtpd_recipient_restrictions is another restriction stage that holds a list of specific restrictions. Other restriction stages that are evaluated prior to smtpd_recipient_restrictions are smtpd_client_restrictions, smtpd_helo_restrictions and smtpd_sender_restrictions (in that order). Restrictions that would normally go in these prior restriction stages can alternately be placed in smtpd_recipient_restrictions. Therefore, some people prefer to place all the smtpd_*_restrictions that would normally go in prior restriction stages into smtpd_recipient_restrictions (in the proper order) and leave the prior stages unconfigured (empty). In our case it is safer to use smtpd_sender_restrictions and smtpd_recipient_restrictions. Let's look at those specific restrictions (tests) we place in smtpd_recipient_restrictions:
1) permit_mynetworks Allows machines listed in "mynetworks" to skip the rest of the tests in this restriction stage (permit = OK). In other words, it exits this stage and is tested in the next stage (smtpd_data_restrictions). Because permit_mynetworks is placed in front of reject_unauth_destination, this means machines in $mynetworks are allowed to relay mail to any domain. Without this, we would only be able to send mail to our own domain(s). If the IP address of the sender is not listed in $mynetworks, the test evaluates to "DUNNO" and continues on to the next test (reject_unauth_destination).
2) reject_unauth_destination This, along with permit_mynetworks is used for relay control. This setting, in essence, means that mail bound for any domain that we have not configured our machine to accept mail for will be rejected. In our case Postfix will use the relay_domains setting (or table) that we configured earlier to determine what domains those are. See http://www.postfix.org/postconf.5.html#reject_unauth_destination for additional details. If the domain is listed in relay_domains, this test evaluates to "DUNNO" and the session is allowed to go on to the next test (if any). Just like "mynetworks", this setting is extremely critical. By placing permit_mynetworks directly ahead of reject_unauth_destination, we are assured that we can send mail to domains other than ours, but we will only accept mail addressed to us from computers outside our network, thus permit_mynetworks and reject_unauth_destination work as a team.
3) reject_unauth_pipelining Rejects bulk mailers that attempt to use pipelining to speed delivery, without checking if it is supported first (non-RFC, common among spammers).
postconf -e "smtpd_recipient_restrictions = permit_mynetworks, reject_unauth_destination, reject_unauth_pipelining"
|
smtpd_data_restrictions: Optional access restrictions that the Postfix SMTP server applies in the context of the SMTP DATA: command. Like smtpd_recipient_restrictions, this is a restriction stage.
1) reject_unauth_pipelining I repeat this setting in smtpd_data_restrictions as it is not always effective when placed in smtpd_recipient_restrictions. I include it in smtpd_recipient_restrictions as I like to place it prior to any policy servers (discussed in a separate document I show you later). Note that there are only a couple of restrictions that make good use of smtpd_data_restrictions.
postconf -e "smtpd_data_restrictions = reject_unauth_pipelining"
|
Postfix content filtering control files:
http://www.postfix.org/header_checks.5.html
/etc/postfix/header_checks and /etc/postfix/body_checks
These files will list certain "strings" of text, and tell Postfix what to do with mail if it encounters these strings in email headers or the body of the message. Sample files are already created for us, with comments explaining what to put in them. You can edit them at your leisure. Note that these files require the use of "regular expression format". "regexp" is something you'll want to learn about in order to live in the *nix world. Get a book or research it on the Net sometime. For an example of an elaborate header_checks file from someone with an attitude, take a look at http://www.geekounet.org/filters/header_checks. But don't blindly follow this, it's only an example! Here is an example of a body_checks file: http://www.securitysage.com/files/body_checks. It would be worth your while to check out http://www.securitysage.com/guides/postfix_uce.html when you have a leisure moment. Specifically http://www.securitysage.com/antispam/hedchek.html and http://www.securitysage.com/antispam/bodchek.html. I recommend against making a large number of changes to any part of Postfix without giving yourself plenty of time to evaluate the effects. Like a turtle, go slow, live long.
Another note about header_checks. Some people wish to get rid of the "Received: from localhost" header by making a rule something like:
/^Received: from localhost \(localhost\.localdomain \[127\.0\.0\.1\]\)/ IGNOREbut get frustrated when it does not seem to work. The reason would be because this header is written after the mail has passed from amavisd-new, but in master.cf we have disabled header_checks and body_checks when mail is returned to Postfix with the override:
-o receive_override_options=no_header_body_checks,no_unknown_recipient_checksso you would have to remove "no_header_body_checks" if you want to work on headers written after the message has left amavisd-new. You can remove the other amavisd-new related header by setting "$insert_received_line = 0;" in amavisd.conf. The downside would be all your costly header_checks and body_checks will be performed again.
header_checks:
Optional: postconf -e "header_checks = pcre:/etc/postfix/header_checks" |
body_checks:
Optional: postconf -e "body_checks = pcre:/etc/postfix/body_checks" |
content_filter: Here's where we tell Postfix to use amavisd-new. http://www.postfix.org/FILTER_README.html
postconf -e "content_filter = smtp-amavis:[127.0.0.1]:10024" |
/etc/postfix/sender_access http://www.postfix.org/access.5.html
We referenced this file in smtpd_sender_restrictions. We use this file to check the sender right at the front door. In this file, we'll list certain senders/domains/IPaddress ranges for special handling. Below are bogus examples, create your own as you see fit. Please read /etc/postfix/sender_access for more information. Although you could use this file for various purposes, considering the way we have set this up in smtpd_sender_restrictions, I suggest using it to either blacklist senders, or allow certain senders to bypass the remaining tests in smtpd_sender_restrictions.
vi /etc/postfix/sender_access |
#Example sender access map file
makeabuck@mlm.tld 550 No MLM thanks
allspam.tld 550 Spam is not accepted here
badguy.net REJECT
justaspamminfool@allspamallthetime.com REJECT
newsletter-fake-domain.com OK
my-bogus-test-domain.com OK
Since this is a hash table, you need to postmap it as usual:
postmap /etc/postfix/sender_access |
Let's take a look at our changes:
vi /etc/postfix/main.cfCheck the contents of the file for errors and repair if needed. You will probably want to edit relay_domains and mynetworks. |
postfix start |
Check that Postfix responds:
telnet 127.0.0.1 25You should see: 220 sfa.example.com ESMTP Postfix (Debian/GNU) hit [enter] a few times; then type: quit to exit |
postfix stop). Make sure you ran newaliases
and all the postmap commands above. Check all the settings in main.cf and master.cf.
There is a nice paper on troubleshooting Postfix at
http://www.postfix-book.com/debugging.html but keep in mind our system is not
ready to relay mail at this point (it will end up in the queue because we
have not yet configured amavisd-new).
Any time you make changes to master.cf or main.cf or to data tables, most (not all) of the time, it is required that you to reload Postfix.
postfix reload |
Now that we have a basic Postfix configuration, http://www.postfix.org/STANDARD_CONFIGURATION_README.html#firewall and http://www.postfix.org/BASIC_CONFIGURATION_README.html are good places to gain a better understanding of some of the settings we used and at this point these READMEs will make more sense. I am personally using this box on an internal network so I have chosen to omit some of the Postfix configuration parameters that may apply to a system that is on the Internet, but Postfix should function fine the way we have it set up so far even if it is on the Internet. Once our system is fully configured you may wish to use the examples in these READMEs to enhance the settings we have made thus far to better suit your needs. Don't send mail through the system just yet, we need to configure amavisd-new first.
By default, our Postfix runs chrooted. If you don't know what that means, it will not matter much at this point. I leave it to you to research what 'chroot' means at some later time. Some of the system files that Postfix needs to run properly were copied to Postfix's chroot jail (/var/spool/postfix) during installation. On occasion, the original files will get modified, and Postfix will complain that the copy it has is not the same as the original. When this happens, you can manually copy the file(s) postfix has complained about to the chroot jail, or we can simply run a script that is supplied with the Postfix source code (called LINUX2) that will once again copy all the files that Postfix needs to where it needs them.
Quick lesson: To change to the Postfix directory shown below, you could type
cd p then hit the [tab] key and
the bash shell will fill the remainder in with the first unambiguous item. If there
is more than one item, you can hit the [tab] key twice and it will list all the available
items.For example, the second line below could be entered as
cd p[tab] e[tab]
c[tab][enter]
cd /usr/local/srcThis next line may be edited for a different version of postfix: cd postfix-2.1.5/examples/chroot-setuppostfix startchmod +x LINUX2cp LINUX2 /usr/binLINUX2cdYou can check if Postfix is happy: postfix check |
At this point, if this server is going to be on the Internet (as opposed to residing on an internal network) you can greatly decrease the load on the server by increasing the amount of mail that is rejected at the front door. You may choose to continue on without reading it right now, and consider using that document a little later, as it is optional (but highly recommended).
Additional anti-UCE settings for our Debian Anti-Spam Anti-Virus Gateway Email Server
|
******************************************************
Configuring amavisd-new:
Index******************************************************
First, make a backup of the amavisd-new config file:
cp /etc/amavis/amavisd.conf /etc/amavis/amavisd.conf-2.4.5-original |
General note. The settings used here are what might be used at small business. A business where the users trust the administrator(s) to make choices for them. In more complex systems, individuals will want their own personal settings. I do not cover that scenario in this document. Suffice it to say that if you need to provide individual settings for a large number of users, the most efficient way to do this would be to use LDAP or SQL to store those settings (dynamic tables). You would need to read the amavisd-new documentation and have some knowledge of LDAP or SQL. You can however provide individual settings to a small number of users by using static tables directly in amavisd.conf, or in plain text files read into amavisd-new via the read_hash() function. There are examples in amavisd.conf. If you use static tables, it is necessary to stop and start amavisd-new after each change.
Next, we'll edit this file, but first let me just mention that this file is a very important part of configuration control in your spam system. There are many settings in here. I won't cover all of them by any means. This file is an abbreviated version of one of those heavily commented config files that some hate, others love. I recommend you return and spend some time reading /etc/amavis/amavisd.conf-sample after this system is all set up, to learn more about just what you can do with amavisd-new. We are going to start at the top of the file and work our way down. Remember, in vi you can search for a string by hitting [Esc], a slash, the text to search for, and optionally a lower case 'n' to find the next occurrence. Once you find what you want, don't forget to use 'i' to start editing. I personally find it much easier to edit this particular file using the WinSCP editor (select the file, hit F4 to edit).
Either: vi /etc/amavis/amavisd.conf
or use the WinSCP editor to edit this file. |
Move to the line that begins with:# @bypass_virus_checks_maps = (1);And uncomment by removing the "#" symbol at the beginning of this line. |
Insure the amavis user and group are set like so:
$daemon_user = 'amavis';
Locate the line that begins with $mydomain:$mydomain = 'example.com';
and change to reflect the actual name of our domain. Next, uncomment # $myhostname and change it to our host's FQDN:$myhostname = 'sfa.example.com';
|
Locate this next line:@local_domains_maps = ( [".$mydomain"] );Comment out that line, then add a new one just like it with all your domains listed in it. Like this: @local_domains_maps = ( [ ".$mydomain", '.example2.com', '.example3.com' ] );
Or like this: @local_domains_maps = ( [qw( .example.com .example2.com .example3.com )] );
Yes, you need those periods there, and don't forget the semi-colon at the end. Note that in Perl, single quotes and double quotes work differently from each other. |
Let's move on to SpamAssassin settings. Amavisd-new calls SpamAssasssin like a sub-routine. SpamAssassin picks up all settings in local.cf, but modified mail as prepared by SpamAssassin is not used, amavisd-new does its own modifications to the original mail. These settings are part of what determines what amavisd-new does with the mail and may include rewriting of the Subject: line, addition of the X-Spam-* lines in the headers and possibly directing amavisd-new to take other action with the mail.
Change$sa_tag_level_deflt = 2.0;to $sa_tag_level_deflt = undef;This will insure all mail addressed to domains in @local_domains will get a spam score in the header, spam or not. Change $sa_tag2_level_deflt = 6.31;to $sa_tag2_level_deflt = 5.0;This low number is assuming your email clients do not automatically discard spam. Set this to 8.0 or higher if they do. The Subject line will be prepended with "Spam> " for any mail that scores above $sa_tag2_level_deflt and is passed to a recipient. Change $sa_kill_level_deflt = 6.31;to $sa_kill_level_deflt = 8.0;On our system, this will trigger the spam to be quarantined if it scores 8.0 or higher. If you plan on deleting the spam, set this at 12.0 or higher. If you only want spam tagged and sent to the recipients (not quarantined at all), set this at 9999 (this would be for ISPs and large companies that configure the MUAs to further process spam). |
/etc/cron.d/amavisd-new and
/etc/cron.daily/amavisd-new.
Note that when these cron jobs are running, the Bayes files will at times be
inaccessible. If you happen to see in a log file that SpamAssassin has timed out,
this could be one possible reason.
Now find:$virus_admin = "postmaster\@$mydomain";The default is fine, but you may wish to change it. Now add a line for banned file notifications: $banned_admin = "postmaster\@$mydomain"; |
Next, locate this section:
$mailfrom_notify_admin = "virusalert\@$mydomain"; $mailfrom_notify_recip = "virusalert\@$mydomain"; $mailfrom_notify_spamadmin = "spam.police\@$mydomain";and change it to: $mailfrom_notify_admin = "postmaster\@$mydomain"; $mailfrom_notify_recip = "postmaster\@$mydomain"; $mailfrom_notify_spamadmin = "postmaster\@$mydomain";Just below this, remove the '#' to uncomment this line: # $hdrfrom_notify_sender = "amavisd-new <postmaster\@$mydomain>";
|
Next, locate # $recipient_delimiter. You set recipient_delimiter
in Postfix and the setting in this file needs to be set similarly. This is
typically set to one of these three options:
$recipient_delimiter = ''; # No recipient delimiter (this is the default)or $recipient_delimiter = '+';or $recipient_delimiter = '-';Change: #$sa_spam_subject_tag = '***SPAM*** ';to $sa_spam_subject_tag = 'Spam> ';The longer version simply takes up too much real estate on the subject line. |
The next line I would like to change begins with:$final_spam_destiny = D_BOUNCE;Change this to: $final_spam_destiny = D_DISCARD; |
$spam_quarantine_to = undef;
and the email with a SpamAssassin kill score of 12 or higher would evaporate
into thin air, or you could configure $sa_quarantine_cutoff_level.
But don't do this; at least not yet. Setting
$spam_quarantine_to = undef; essentially
means "we don't have a spam quarantine area so I guess I have to just dump the
mail". This is something you probably should not implement at first; at least not if you
enjoy being employed.
If you decide to do this, I suggest you wait
a month or so to get a feel for the system (and let the
Bayes
database initialize). You can eliminate about
90% of the email that ends up in the "spambin" by using this suggestion.
Next, locate the line that looks like this: $virus_quarantine_to = 'virus-quarantine'; I suggest you do one of three things here. You can leave this alone, and viruses will be quarantined on the spamfilter box, or you can set this to: $virus_quarantine_to = undef;and provided we configure a virus scanner, all caught viruses will disappear, or you can send caught viruses to a regular mailbox: $virus_quarantine_to = "virii\@$mydomain";
You can choose to use "spambin" for the quarantine area for all spam, viruses and email with banned attachments, but I recommend you create separate mailboxes for each. |
Next, locate the $banned_quarantine_to line and configure these
three lines in this manner: $banned_quarantine_to = "banned\@$mydomain"; $bad_header_quarantine_to = "banned\@$mydomain"; $spam_quarantine_to = "spambin\@$mydomain"; Make sure you have mailboxes for these two addresses on a destination server. This is where you will review quarantined email, and if ham is found, will forward the ham to the proper recipient. |
Locate this block inside the $banned_filename_re = new_RE section:qr'.\.(exe|vbs|pif|scr|cpl)$'i, # banned extension - basic # qr'.\.(exe|vbs|pif|scr|cpl|bat|cmd|com)$'i, # banned extension - basic+cmd # qr'.\.(ade|adp|app|bas|bat|chm|cmd|com|cpl|crt|emf|exe|fxp|grp|hlp|hta| # inf|ins|isp|js|jse|lnk|mda|mdb|mde|mdw|mdt|mdz|msc|msi|msp|mst| # ops|pcd|pif|prg|reg|scr|sct|shb|shs|vb|vbe|vbs| # wmf|wsc|wsf|wsh)$'ix, # banned ext - longIf you would like to greatly extend the types of attachments amavisd-new bans, you might want to comment out, and uncomment some lines like so:
# qr'.\.(exe|vbs|pif|scr|cpl)$'i, # banned extension - basic
# qr'.\.(exe|vbs|pif|scr|cpl|bat|cmd|com)$'i, # banned extension - basic+cmd
qr'.\.(ade|adp|app|bas|bat|chm|cmd|com|cpl|crt|emf|exe|fxp|grp|hlp|hta|
inf|ins|isp|js|jse|lnk|mda|mdb|mde|mdw|mdt|mdz|msc|msi|msp|mst|
ops|pcd|pif|prg|reg|scr|sct|shb|shs|vb|vbe|vbs|
wmf|wsc|wsf|wsh)$'ix, # banned ext - long
Just edit out the attachment types you would like to receive or edit in
any others you would like to ban. Each vertical bar means "or". I would
add bin|drv|mht|ocx|ovl| If you receive
mail from people using Outlook in Rich Text mode, make sure you
don't block |tnef When an email that has one of these attachments comes into the system, the entire message gets placed in quarantine and the sender and postmaster get notified. Using this feature of amavisd-new goes a very long way toward preventing email borne viruses from entering your network. This method should not be your only line of defense however, just another tool in your arsenal. |
Next, locate the # ENVELOPE SENDER SOFT-WHITELISTING / SOFT-BLACKLISTING section. |
If using vi, save and exit the file with [Esc]:wq Then
make a backup:cp /etc/amavis/amavisd.conf /etc/amavis/amavisd.conf-18aug06I have a habit of using the date for my backups. |
If you have a powerful machine with sufficient RAM (512MB or more), at some time in the future you may want to experiment with increasing the number child processes that amavisd-new spawns. More child processes means that amavisd-new can process more mail in a given amount of time (up to a practical limit). See http://www.ijs.si/software/amavisd/amavisd-new-magdeburg-20050519.pdf. If you configure your system to use more instances of amavisd-new, allocate at least 40MB for each additional instance. It you wanted to double the number of child processes from 2 to 4, you would edit amavisd.conf and change:
$max_servers = 2;to
$max_servers = 4;Then edit master.cf and change:
smtp-amavis unix - - - - 2 smtpto
smtp-amavis unix - - - - 4 smtp
Amavisd-new will be the biggest bottleneck in the system. On a busy server you will probably want 2GB RAM so you can accomodate somewhere around 15 $max_servers.
Our system will log to both /var/log/mail.log and /var/log/mail.info. We don't really need the log duplicated to /var/log/mail.info:
Optionally disable logging to /var/log/mail.info:vi /etc/syslog.conf
and comment out the mail.info line, like so: #mail.info -/var/log/mail.info |
There are files that act as templates (that can be modified) that are used as the basis for the text used in email notifications. Normally these templates are stored inside the amavisd-new file itself, but it is possible to copy this text into external files, and then place a setting in amavisd.conf that uses these external files. The typical Debian installation is configured to use these external files but I have commented out the setting in our custom amavisd.conf. The problem is, the external files provided with the Debian package are for the old version of amavisd-new and are somewhat dated. If you don't plan on customizing any of the templates, you can simply leave the external files disabled, or you can replace them with newer versions, then customize them if desired:
|
Optional: If you would like to enable the external template files so you may customize them in the future (English only): vi /etc/amavis/amavisd.conf
and uncomment the line: # read_l10n_templates('en_US', '/etc/amavis');
|
Let's give amavisd-new a little test run (amavisd-new must be stopped before running it in debug mode):
amavisd-new stopamavisd-new debug |
After just a few moments, if you have something misconfigured; amavisd-new will tell you. At this point don't worry about the razor2 error if you get it. If you have an error in /etc/amavis/amavisd.conf, it will give you a line number and a brief explanation. Fix anything wrong. This will mean reading closely any error messages, and possibly reading the files in /usr/local/src/amavisd-new-2.4.5. There are friends at the AMaViS mailing lists waiting to help. https://lists.sourceforge.net/lists/listinfo/amavis-user. If everything is OK, and it should be, you will see a lot of lines that look like text log entries (exactly what debug mode does - logs to the screen), and near the bottom you will see "Parent ready for children". You can ignore the lines above this that talk about things like "INFO: no optional modules:" and "No decoder for" and "Cache not available", these are not errors, they are simply informational. FYI, each time amavisd-new starts up normally on your box, all these lines will be recorded in the system log files.
| Use the [Ctrl]+c key combination to exit (kill) amavisd-new debug. |
If you are interested (and I realize this link may unnecessarily give you information overload at this point), there is one issue with quarantining mail in the manner I suggest above. The author of amavisd-new (Mark Martinec) suggests using Plus Addressing as an alternative. http://verchick.com/mecham/public_html/spam/plusquarantine.html
******************************************************
Pyzor, Razor and SpamAssassin configuration:
Index******************************************************
Pyzor configuration.
chmod -R a+rX /usr/share/doc/pyzor /usr/bin/pyzor
/usr/bin/pyzord This next line will have to be modified if you have a different version of python. Try locate pyzor (may need to run
updatedb first)chmod -R a+rX /usr/lib/python2.3/site-packages/pyzor
(It might be):
chmod -R a+rX /usr/lib/site-python/pyzor
Here we supply the IP address of the Pyzor server to Pyzor (for both the 'root'
and 'amavis' users). This will create a .pyzor directory in both user's
home directories, and place the server's IP address in a 'servers' file therein:
pyzor discover
Test the pyzor server for a response: pyzor pingsu amavis -c 'pyzor ping'More than likely, you will get TimeoutError:. I recommend using a server other than the one 'pyzor discover' gives us:
echo "82.94.255.100:24441" > /var/lib/amavis/.pyzor/servers
Then test again: su amavis -c 'pyzor ping'
If in the future the IP address of the server changes, you will need to run the two 'pyzor discover' commands again. I suggest you subscribe to http://lists.sourceforge.net/lists/listinfo/pyzor-announce. |
You will need a sample spam to feed to spamassassin:
cd /root
|
amavisd-new -d config debug-sa you would notice that amavis
expects to find programs and configuration files in certain places. If you were to run
spamassassin -D config </root/sample-spam.txt as root you would notice that
root expects to find the same things, but it expects to find some of them
somewhere other than where the amavis user expects. SpamAssassin is designed to enable each
user to have their own settings and data. This section will make both users happy,
and the reason we want to do this is because if we are debugging SpamAssassin
or Razor or Pyzor or DCC, we want to be able to do so with
spamassassin -D </root/sample-spam.txt rather than amavisd-new debug-sa
because we don't want to shut amavisd-new down every time we need to debug one of
those programs. A better way to debug SpamAssassin is to run the program as the
amavis user like so: su amavis -c 'spamassassin -D </root/sample-spam.txt'
spamassassin -D </root/sample-spam.txt
|
http://www.dcc-servers.net/dcc/firewall.html gives an example of a Cisco router access list entry. Keep in mind we are not running a DCC server. Also http://www.oddquad.org/linux/anti-spam.html#providefirewallaccess will give you some idea of what we are after.
We are going to make a backup of the original installation files then copy some of them to where the amavis user expects to find them.
cp -ir /root/.spamassassin /root/.spamassassin-backup
Please answer 'n' to "overwrite?": cp -ir /root/.spamassassin /var/lib/amavis
|
We are going to make a symbolic link, so we only have to deal with one user_prefs file for both the root and amavis users.
rm /root/.spamassassin/user_prefsln -s /var/lib/amavis/.spamassassin/user_prefs /root/.spamassassin/user_prefs |
If you run
sa-learn --force-expire
or spamassassin --lint -D or other
spamassassin commands from the root
account, SpamAssassin may change the owner of the Bayes files to 'root'.
If it does, amavis will no longer be able to read those files.
You would need to run
chown -R amavis:amavis /var/lib/amavis
to regain ownership.
In general, if you do any spamassassin maintenance from the
command prompt as root, the best thing to do is run
chown -R amavis:amavis /var/lib/amavis afterwards;
just to make sure.
You can avoid these problems by remembering to run
spamassassin commands as the amavis user. For example
su amavis -c 'sa-learn --sync --force-expire'
This next section configures Razor; sets the elusive "razorhome" and makes both root and amavis happy in their attempts to figure out "where in the heck are the Razor2 configuration files"?
cd
If you get an error, you may need to run the 'razor-admin register' command more than once. Don't worry about it if /etc/razor/razor-agent.conf does not exist. Now edit root's razor configuration file: vi /root/.razor/razor-agent.conf
and change the line:
debuglevel = 3
to:
debuglevel = 0
Obviously -zero- not -oh-; Save and exit the file. OK, now copy root's .razor directory and files to the amavis user's home directory: cp -r /root/.razor /var/lib/amavischown -R amavis:amavis /var/lib/amavis |
Now we will modify SpamAssassin's main configuration file:
vi /etc/spamassassin/local.cf
And insert the lines:
bayes_path /var/lib/amavis/.spamassassin/bayes
This insures both the root and amavis users use the same files and do not have to guess where they are, and whitelists our spambin. lock_method flock is used
when the Bayes data resides on the local hard disk and is non NFS.
Optional: Since there is a script that runs each day to --force-expire old Bayes tokens "/etc/cron.daily/amavisd-new" (make sure there is if you use this setting!), we can set:
bayes_auto_expire 0
Optional: Some people believe auto-whitelist is more of a liability than an asset:
use_auto_whitelist 0
Possibly optional, possibly not: Depending on your setup, it might be necessary to explicitly set internal_networks and trusted_networks. The trust path tells spamassassin which clients are not trusted. See http://wiki.apache.org/spamassassin/TrustPath and this thread:
# explicitly set our internal_networks (might be the same or similar to mynetworks)
|
dns_available yes .
With SpamAssassin version 3.1.x, additional configuration is needed:vi /etc/spamassassin/v310.preTo enable the ability to use DCC, uncomment the line:
#loadplugin Mail::SpamAssassin::Plugin::DCC
|
|
If you are using SpamAssassin 3.1.1 or greater
you can use the new sa-update feature. If you want to run sa-update without using GPG, run:
sa-update --nogpg
Optionally, you may first import the gpg key:
cd /etc/spamassassin
Then simply run: sa-update
You should find the new rules in /var/lib/spamassassin/<SA version>. You should also run 'amavisd-new -d config debug-sa' and verify SpamAssassin is locating all of its rule sets in /var/lib/spamassassin/<SA version>. You should also run:
su amavis -c 'spamassassin --lint'
after an update, and you should be aware you must reload amavisd-new after the update in order for the new rules to be used. It is important that sa-update completes without error. Optionally:
crontab -e
and insert (on the first available blank line): 05 05 * * * /usr/bin/sa-update && /usr/bin/spamassassin --lint && /etc/init.d/amavis restart
During the time amavisd-new is restarting, mail cannot be delivered to it. Postfix will complain "connect to localhost[127.0.0.1]: Connection refused". Postfix will defer this mail (for about 15 minutes). To speed things up, an impatient person may run 'postfix flush' to flush the deferred queue, but I would not. |
chown -R amavis:amavis /var/lib/amavis
|
If you would like to see if SpamAssassin and Razor are working, run spamassassin with the debug (-D) option (run this a couple times):
spamassassin -D </root/sample-spam.txt
|
If you debug spamassassin as user root I recommend you refresh file ownership every time:
chown -R amavis:amavis /var/lib/amavis
|
A note on Bayes: to better train the Bayes database we need to give it some examples of ham. The problem is,
sa-learn --ham /path/to/hamfiles
works on files that reside on the spamfilter. We don't keep mail on the spamfilter
however. This problem can be solved by using your email client (MUA) to individually
save messages in .EML format. Look for this option among the menu choices or 'Save As'. Edit
a sample .EML file to insure the file looks like plain text. If it is full of
garbage characters, then it cannot be used. Use WinSCP to copy your ham collection to
an empty folder on your spamfilter. Then run sa-learn. Pick messages that
have some substance to them so the database has something to work with. Use this
especially when you receive a message that is tagged as spam, but is in fact ham.
Autolearning (bayes_auto_learn) is turned on by default so the system will have no problem gathering
plenty of spam. I suggest you only feed it additional examples of ham to begin with.
In the future you will want to feed it samples of low scoring spam.
SpamAssassin has the ability to store its Bayes data in a MySQL database. If you are familiar with MySQL (or not) and you wish to improve performance, you should consider using it. Here is my HOWTO to get you going: http://verchick.com/mecham/public_html/spam/debian-spamassassin-sql.html
******************************************************
Installing DCC:
Index******************************************************
DCC is available from the Debian archives, but we will get it from the author and compile it from the source code. This is one reason we installed a C compiler during installation. Installing it from source is a good exercise and we have better control over how it installs. Installing from source allows us to customize the installation for use with amavisd-new. Note that as of version 1.3.0 of DCC http://www.commtouch.com/ has exclusive marketing rights for DCC. If you resell anti-spam solutions that use DCC and you do not provide your DCC data to the public, you will need to pay for DCC. Please read the license.
cd /usr/local/srcwget http://www.dcc-servers.net/dcc/source/dcc-dccproc.tar.Ztar xzvf dcc-dccproc.tar.Z
Change to the dcc subdirectory by using the [tab] key command completion shortcut as shown, then ./configure: cd dcc-dccproc- [tab][enter]
./configure --with-uid=amavis && make && make install
The double ampersands let you run those 3 commands on one line. You will see 'done' if all goes well. Place a link to cron-dccd in our path: cd
Test our installation with: cdcc info
We should get 'requests ok' from the servers. The instructions say to run cron-dccd each day to clean things up, so we will do that. crontab -e
and insert (at the top): 43 11 * * * /usr/bin/cron-dccd
Make sure you have a carriage return at the end of the line, then save the file (same command as vi) |
Now test it with SpamAssassin:
su amavis -c 'spamassassin -D </root/sample-spam.txt'
|
If one of them is broken, you would edit
/etc/spamassassin/local.cf and disable the offending party
as needed: use_razor2 0
use_pyzor 0
use_dcc 0
I hope you got them all working because they really are valuable resources in the fight against spam. I have noticed over the last year that fewer and fewer people are submitting spam to the pyzor servers with the result being fewer and fewer messages are tagged by pyzor.
Now give it the full blown amavisd-new and SpamAssassin test:
amavisd-new stopamavisd-new debug-sa |
You should see nothing but happiness (except for the "No $unfreeze, not using it" type stuff).
Please stop amavisd-new at this time with [Ctrl]+c.
Normally SpamAssassin runs the 'dccproc' program for each message that it processes. If you have 30MB RAM to spare, you may benefit from running the daemonized version of the DCC client 'dccifd'. I personally recommend you have a system with at least 384MB RAM to do this. Note that if you were unable to get DCC working before, enabling dccifd will not solve the problem and should not be attempted.
Optional to enable dccifd:vi /etc/spamassassin/local.cfand insert: dcc_home /var/dccsave and exit, then: vi /var/dcc/dcc_confand change DCCIFD_ENABLE=offto: DCCIFD_ENABLE=onThen change: DBCLEAN_LOGDAYS=14to: DBCLEAN_LOGDAYS=1save and exit. If you choose to allow logging, cron-dccd should delete old log files when it runs. Keep your eye on the files that accumulate in the /var/dcc/log directory. It's your choice, but I personally don't want to monitor the DCC logs, so I turn off logging altogether by deleting the log directory and commenting out the logdir entry in dcc_conf: rm -r /var/dcc/logvi /var/dcc/dcc_confand comment out: DCCIFD_LOGDIR="$DCCM_LOGDIR"We will use a supplied script (rcDCC) to automatically start dccifd when we boot up: cp /var/dcc/libexec/rcDCC /etc/init.d/adccupdate-rc.d adcc defaultsThere is a bug in DCC version 1.3.31 or older. We can work around the problem by editing /etc/init.d/adcc and commenting out one line (that we don't need). This file is read only so when you save it, you will have to save it using [Esc]wq! : vi /etc/init.d/adcc
Then comment out this line: # $DCC_LIBEXEC/start-dccm $DEBUG
Save and exit the file, then update file permissions: chown -R amavis:amavis /var/dccBecause we enabled dccifd in dcc_conf, we can start up dccifd by running the script: /etc/init.d/adcc startIf you deleted the log directory, you can expect an error message: "log thresholds set with -t but no -l directory". Now test that SpamAssassin finds dccifd:
su amavis -c 'spamassassin -D </root/sample-spam.txt'
You should see debug: DCCifd: got response: |
******************************************************
Local DNS cache:
Index******************************************************
Having a local DNS cache is highly recommended provided you currently do not have a DNS server or other server that provides a DNS caching function on your network and you have sufficient available memory. I have found that using a broadband router (or similar) as a primary name server does not provide the same functionality a true DNS server provides. Name server settings are located in
/etc/resolv.conf. If you
choose not to use a local DNS cache then at
least use a real DNS server as your primary. I have seen
SpamAssassin time out on RBL lookups if it cannot perform DNS queries quickly
enough. This can have a big effect on the final spam score.
I will not pretend that I understand the intricacies of the bind9
program that we will install. We will configure bind9 to be a caching only
name server with the option of forwarding requests to another server.
You may also consider providing the service we install on this machine
to other machines on your network. Note that you will need to modify
our local firewall (iptables) if you choose to provide this service to
other machines:iptables -A FIREWALL -p udp -m udp --dport 53 -j ACCEPTiptables -A FIREWALL -p tcp -m tcp --dport 53 -j ACCEPT(or for enhanced security, only allow access from clients on our network):
iptables -A FIREWALL -p udp -m udp -s 222.222.222.222/24 --dport 53 -j ACCEPTiptables -A FIREWALL -p tcp -m tcp -s 222.222.222.222/24 --dport 53 -j ACCEPTSee the "Create Firewall Rules:" section above.
The bind9 manual is available at http://www.bind9.net/manual/bind/9.2.4/Bv9ARM.html and I wish to thank this document for much of the information listed here: http://www.falkotimme.com/howtos/perfect_setup_debian_sarge/index.php
Previous versions of this document included instructions to install djbdns (dnscache) but I no longer recommend that program. It seems to work great at first, but over time I was getting errors in my Postfix log that appeared to stem from an unresponsive DNS server. Since removing djbdns and installing bind9, I have not had those errors. Note that bind9 does use considerably more memory than djbdns.
|
Please read the notes above to determine if a local caching DNS server is required.
If you have used an earlier version of this document and you are
currently using djbdns (dnscache), it MUST first be disabled
and our nameserver configuration must be changed so it no longer uses 127.0.0.1
as a nameserver entry before we can install bind9. Obviously you can only have
one type of nameserver daemon running. If you do not have djbdns installed, and you
are not running any other type of name server daemon on this machine, you can skip
the instructions in this particular box. The first thing we need to do is edit /etc/resolv.conf and change the nameserver entries to point to valid name servers (either on our network, or at our ISP): vi /etc/resolv.confand change the nameserver entries(s):
nameserver 444.444.444.444
Test that we are able to resolve host names: dig yahoo.com
You should see valid data, and the output will also tell you which name server was used to find the information: ;; SERVER: 444.444.444.444#53(444.444.444.444) Make sure this is your primary name server as configured in /etc/resolv.conf. Once this is working, we must tell Postfix to use the new name servers: LINUX2
You can see that dnscache is currently running a name server service by running: lsof -i | grep :domain
Now we will get rid of djbdns: rm /service/dnscache
Wait about 5 seconds, then make sure dnscache is no longer running: lsof -i | grep :domain
You can also run dig yahoo.com again if you
would like to make sure you are still able to resolve host names. If for some
reason you want to undo these changes and continue to use djbdns, you can
reinstall it using the original instructions located at
http://verchick.com/mecham/public_html/spam/djbdns.html You would want to start from
the line with the command "build-djbdns"
|
Now we install bind9:
For consistency, we will install version 9.2.4 of bind9:
apt-get -t stable install bind9
For security reasons we want to run BIND chrooted so we will perform the following steps: /etc/init.d/bind9 stop
Edit the file /etc/default/bind9 so that the daemon will run as the unprivileged user 'bind', chrooted to /var/lib/named: vi /etc/default/bind9
Modify the line: OPTIONS="-u bind" so that it reads: OPTIONS="-u bind -t /var/lib/named"
Create the necessary directories under /var/lib: mkdir -p /var/lib/named/etc
Then move the config directory from /etc to /var/lib/named/etc: mv /etc/bind /var/lib/named/etc
Create a symlink to the new config directory from the old location (to avoid problems if bind is upgraded in the future): ln -s /var/lib/named/etc/bind /etc/bind
Make null and random devices, and fix permissions of the directories: mknod /var/lib/named/dev/null c 1 3
We need to modify the startup script /etc/init.d/sysklogd of sysklogd so we can get important messages logged to the system logs: vi /etc/init.d/sysklogd
and replace: SYSLOGD=""
with: SYSLOGD="-a /var/lib/named/dev/log"
Save and exit the file, then restart the logging daemon: /etc/init.d/sysklogd restart
We can start up bind9 at this point: /etc/init.d/bind9 start
Let's see if the service is running: lsof -i | grep :domain
It is also imperative that after each change we make, we look in our syslog to see if bind9 reported any errors. Here we use less
to view syslog. Once we are viewing the file, use an uppercase 'G'
to go to the bottom of the file (and a lowercase 'q' to quit):less /var/log/syslog
We are setting up bind9 as a local caching only name server (later we will optionally configure it as a forwarding server). Here we add some additional security measures that prevent unauthorized machines from using our name server: vi /etc/bind/named.conf.options
On the line below "directory" we want to add a line that restricts use of our name server to the network our spamfilter is on. Place a [Tab] in front of the entry so it lines up with the other entries. You can add more than one network here if you like. Place a ";" (semicolon) after each network. Note that if you actually want to allow other clients to connect to our name server, as explained in the notes above you would also have to modify IP tables to allow this. allow-query {222.222.222.222/24;};
Save and exit the file, then I would restart bind9 and check that it is running: /etc/init.d/bind9 restart
Optionally configure bind9 as a forwarding server. Bind9 as we have it configured now will first query the root servers for hints when needed. I prefer to forward queries to another name server instead. There are advantages and disadvantages in doing this, but I prefer it. It is absolutely imperative that any name servers listed here are known to work from our spamfilter. These will almost certainly be the primary and secondary servers you currently have configured in /etc/resolv.conf (not 127.0.0.1, and not the IP address of the local machine) or your ISP's servers. Add the 'forwarders' entry just below the 'allow-query' entry we just made: vi /etc/bind/named.conf.options
and add: forwarders {444.444.444.444; 555.555.555.555;};
To never query the root servers, optionally add (personally I do add this): forward only;
Save and exit the file, then once again I would restart bind9 and check that it is running: /etc/init.d/bind9 restart
And once again, check for errors: less /var/log/syslog
Once it is determined bind9 is functioning, you can change the primary nameserver in /etc/resolv.conf: vi /etc/resolv.conf
and replace the entry for the primary nameserver (the first one listed): nameserver 444.444.444.444
to the IP address of this machine (our real IP address, not 127.0.0.1): nameserver 111.111.111.111
Save and exit the file, then test that we are able to resolve host names: dig yahoo.com
You should see valid data, and the output will also tell you which name server was used to find the information: ;; SERVER: 111.111.111.111#53(111.111.111.111) Make sure this is shows our IP address as configured in /etc/resolv.conf. Now we can tell Postfix to use the new name server: LINUX2
|
******************************************************
Test the Installation:
Index******************************************************
Let's start off fresh:
reboot
logout |
If you happen to see the message "SpamAssassin Mail Filter Daemon: disabled, see /etc/default/spamassassin"; please ignore it, we do NOT want to enable spamd.
Other mail servers will need to find your server. Set up name resolution of sfa.example.com to its IP address. How this is done depends on your environment. Basically, you need to make sure that any box that needs to talk to this mail server can resolve its name, either through appropriate DNS server entries somewhere, or local "hosts" files on those machines.
Now let's see if this box is actually an email server!
Add a new entry in the hosts file on your Windows computer that points to the spamfilter:
111.111.111.111 sfa.example.com
You should already have a mailbox set up for "spambin". Create a user in your email client (MUA) for "spambin" so you can send and receive email from that account if you have not already done so. Temporarily set your email client's SMTP (Outgoing) server to sfa.example.com for the "spambin" account so any mail it sends gets sent back through the spamfilter. It is required however to whitelist "spambin" so the email doesn't end up back in the "spambin". This is the problem with forwarding email from the "spambin" (or "banned" or "virii") client. Any email you forward out of these mailboxes may end up back in the quarantine. Once we are done testing, you should set up the "spambin", "banned" and possibly "virii" email clients to send their email directly to the Exchange server. Of course you would never actually forward viruses to end users or open any viruses in this quarantine, but you may occasionally find legitimate mail that was quarantined due to a banned attachment. You should obviously have virus protection on the client machine.
Send yourself an email from the "spambin" account. Wait a minute, and then "Get new mail" or "Send and Receive" or whatever. If you don't get the email "spambin" sent you, obviously "Houston, we have a problem". You might check the mail log on the spamfilter for clues:
less /var/log/mail.log
If you would like to see the log go by in real time:
tail -f /var/log/mail.log
I use this command extensively, and I suggest you use it after any changes are made to Postfix or amavisd-new. This will tell you if your mail system is receiving and processing mail, or if your change just killed the system.
and/or use
amavisd-new stopamavisd-new debug-safor more detail.
You can increase the level of detail reported by adjusting $log_level in amavisd.conf. Remember to set it back to 0 when finished with your debug session.
cd /var/mail
Then
less any files you may find there. There may be a file
or two that ended up there before Postfix was configured.
Also try the commands mailq
and qshape (and qshape deferred ) to see if there is
mail stuck in the queue. Use amavisd-new debug
while you are sending mail through the system to help provide clues to the problem.
Open another terminal window and run mailq while the
other window is running amavisd-new debug . If you
made changes to configuration files and want to flush the queue, try
postconf -f and if that does not work try
postsuper -r ALL . See
http://www.postfix.org/postsuper.1.html. Insure that the relay_domains parameter
has been configured correctly. Inspect the /etc/postfix/transport
file for errors, and make sure you run
postmap /etc/postfix/transport every time you make changes.
If you have problems, the most likely reason is a typographical error, or a wrong IP or network address in one of the configuration files. Hopefully, you know Unix/Linux is cAse senSitiVe. Maybe you accidentally skipped one of the configuration steps. Some of the steps cannot be performed more than once however, so be careful. Remember, read /var/log/mail.log as a first step.
If you do get the email back, change whatever setting or use whatever command or tool you have that will enable you to look at the full headers of the email message. Examine the headers and behold your creation. If you sent an empty message with no subject, Pyzor might just flag it as spam!
As time goes by be sure to keep an eye on the number of files and temporary directories that are created in the /var/lib/amavis directory and its subdirectories. If there are a large number of files and/or temporary directories it would indicate some sort of problem. If you choose to keep quarantined messages on this box instead of sending them to spambin@example.com then they should accumulate in the virusmails directory; that would be considered normal. I have seen on many occasions people report that amavisd-new fails to delete temporary directories (their name begins with amavis-20) and there seems to be a lack of understanding as to the reason this occurs. It may possibly indicate a problem with one of the shared libraries that amavisd-new uses (uulib zlib etc.) or a problem with Perl itself. See http://www.ijs.si/software/amavisd/#faq-gen. I have seen reports of people writing scripts that delete these temporary directories on a regular basis to work around the problem but this is only sweeping the underlying problem under the rug.
Whenever you edit a few certain files like the hosts file on the spamfilter, we need to supply Postfix with an up to date copy of it or will complain: warning: /var/spool/postfix/etc/hosts and /etc/hosts differ
LINUX2postfix check |
Congratulations, you did a great job. It's either time for that 4th cup of coffee, or maybe just finish the next few sections and call it a day, because in many respects, we are just getting started.
Huh?
******************************************************
Installing ClamAV:
Index******************************************************
This entire section is optional but recommended. It would be a good idea to get ClamAV configured and running, even if you choose to disable it. It's reasonably easy to turn it on and off. If you use this product regularly, consider a donation. It's also a good idea to read the mailing list archives http://lurker.clamav.net/list/clamav-users.en.html and join the clamav-announce mailing list http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-announce so you will be notified of new releases. Keep in mind that after a new version is released, it may take a couple days for the Debian package maintainer to update Debian.
Read http://verchick.com/mecham/public_html/spam/clamav-amavisd-new.html
if you would like to gain a better understanding of what we are doing here.
We need to install a Debian Volatile source in our sources.list:vi /etc/apt/sources.list
And insert: deb http://volatile.debian.net/debian-volatile sarge/volatile main
Save and exit the file. Note: you may want to choose a source that may be closer to you, look here. As usual, you must run 'apt-get update': apt-get update
Then install clamav, and clamav-daemon from Volatile: apt-get -t sarge install clamav clamav-daemon clamav-freshclam
Now, very important, add the clamav user to the amavis group: gpasswd -a clamav amavis
|
If (and only if) you are using version 0.90 or newer you may need to modify freshclam.conf:vi /etc/clamav/freshclam.conf
and make sure you have a line like this (insert or modify as needed): NotifyClamd /etc/clamav/clamd.conf
|
Save and exit freshclam.conf then configure amavisd.conf so amavisd-new will use ClamAV:
vi /etc/amavis/amavisd.conf
To enable virus scanning, comment out the line: @bypass_virus_checks_maps = (1); # uncomment to DISABLE anti-virus code
Locate the line: @av_scanners = (
Uncomment these 4 lines, then make sure the value after CONTSCAN reads as follows:
['ClamAV-clamd',
\&ask_daemon, ["CONTSCAN {}\n", '/var/run/clamav/clamd.ctl'],
qr/\bOK$/, qr/\bFOUND$/,
qr/^.*?: (?!Infected Archive)(.*) FOUND$/ ],
While you are at it, if you would like to clean up amavisd.conf a little, you may optionally
delete all the virus scanners between ClamAV and BitDefender. We want to keep
BitDefender. It would be a very good idea to make a backup copy of amavisd.conf before
you do this.
|
/var/run/clamav/clamd.ctl value shown above must match the LocalSocket
parameter in /etc/clamav/clamd.conf Change it here if necessary to match what is in /etc/clamav/clamd.conf.
The configuration file for ClamAV 0.90.1 is
/etc/clamav/clamd.conf.
The freshclam daemon (or cron job) is set to check for, and download if necessary,
new virus definitions 12 times per day.
The configuration file
for freshclam is /etc/clamav/freshclam.conf
You can check the log files at
/var/log/clamav/freshclam.log and
/var/log/clamav/clamav.log
The startup script for freshclam is
/etc/init.d/clamav-freshclam
and for clamd it's /etc/init.d/clamav-daemon
If you chose cron for database updates, the file is
/etc/cron.d/clamav-freshclam. You can also research
additional clamd.conf configuration options by running man clamd.conf
Let's reboot (it might make a difference), and then test the system:
rebootlogout
As an alternate to rebooting, you can stop and start clamd and amavisd-new: amavisd-new stop
Once the system comes back up, we need to test ClamAV. To do so, we can simply watch the mail.log go by as we send the Eicar test virus through the system: tail -f /var/log/mail.log
Send the Eicar test virus through as described below. After you are finished sending the virus through, use [Ctrl]+c to return to the shell prompt. |
Once this machine is up and running in a production environment you should only reboot when necessary. There are many processes writing to the disk in this system and killing those processes is sometimes not a good idea.
Now you need to send a test virus to yourself, like eicar.com.txt from http://www.eicar.org/anti_virus_test_file.htm. Note: rename eicar.com.txt to something like eicar.txt because amavisd-new bans files with double extensions. Attach eicar.txt to the email you send. Don't use eicar.com or eicar_com.zip. Of course you would have to temporarily disable your desktop antivirus software first. You can use
tail -f /var/log/mail.log so you can see what happens.
If ClamAV is not working you may get "Can't connect to UNIX socket". This could indicate a couple of things. The LocalSocket parameter in /etc/clamav/clamd.conf must be the same as the one in /etc/amavisd/amavisd.conf as noted above. The clamav user must also belong to the same group that the amavis user belongs to so both users can write to the UNIX socket file, in our case "clamd.ctl".
I have a funny story. I had this all working fine but it appeared that clamd thought that the eicar_com.zip file was "OK", in other words, it scanned it, but did not detect that is was a virus. It turned out my Internet proxy (WinProxy) had removed the virus from inside the file before I scanned it! It took me an hour to figure it all out.
If you have problems that you can't seem to solve, or you are upgrading to a new major release, completely uninstall the program, and then try another version if available:
apt-get --purge remove clamav clamav-base clamav-daemon clamav-freshclam libclamav1
Watch for errors as the program is being uninstalled, you may have to manually remove any clamav directories that dpkg failed to remove (rm -r). Be extremely careful here!.
If you just can't get it working, uncomment
@bypass_virus_checks_maps
in /etc/amavis/amavisd.conf to disable virus scanning
(then maybe go shopping for another antivirus vendor).
Also: make sure everything works AFTER you reboot. Don't leave ClamAV in a
non-functional state if you have virus scanning enabled. Amavisd-new will not work properly.
Keep an eye on /var/log/clamav/freshclam.log
and /var/log/clamav/clamav.log. You need
to look for error messages in these files. You may have an error or two when the
program is first installed; this is probably OK and may be due to things happening
out of sequence. Check the logs for a couple days and make sure the database
updates do not fail and that Clamd is notified of updates.
When it is time to upgrade to a new version of ClamAV:
Run 'apt-get update'.
If you are upgrading to a MAJOR new version, completely uninstall ClamAV as described above.
Install the new version of clamav and clamav-daemon.
Run 'gpasswd -a clamav amavis'.
I suggest rebooting, if possible.
Run 'tail -f /var/log/mail.log', and send a test message containing the eicar "virus" through.
There is a third party http://www.sanesecurity.com/clamav/ that maintains additional anti-phishing and scam mail signatures. If you would like to add these I have customized for Debian one of the scripts used to download the signatures:
Optionally use additional anti-phishing and scam signatures:
cd /usr/sbin
Always remember that stuff you download can be forged. I suggest you glance through the script:
less scamp.sh
Then run the script (if it looks like it's not a script that destroys the planet):
scamp.sh
It should have produced something like: Updated: phish.ndb Thu Nov 23 08:48:38 MST 2006 Updated: scam.ndb Thu Nov 23 08:48:39 MST 2006 and if you look at the list of clamav signatures, you should see these have been added:
ls -l /var/lib/clamav
Per the instructions, add an entry to your crontab:
crontab -e
Set MM to three minutes from now:
MM */4 * * * /usr/sbin/scamp.sh
|
At some point in the future you may get tired of wading through all the comments in amavisd.conf. Here's how to remove them:
Optionally remove most comments from amavisd.conf:cp /etc/amavis/amavisd.conf /etc/amavis/amavisd.conf-verbose
grep -vE '^$|^#' /etc/amavis/amavisd.conf > /etc/amavis/amavisd.conf-temp
cp /etc/amavis/amavisd.conf-temp /etc/amavis/amavisd.conf
|
******************************************************
Tweaking Notification Settings:
Index******************************************************
This may be a good time to experiment with some of the notification settings in amavisd.conf. Change some of these settings and send yourself email containing viruses, banned attachments, spam, and attachments with double extensions. Do one thing at a time and take notes on what happens and who gets notified. Remember to stop and start (or debug) amavisd-new after each change. Settings to play with are:
$final_virus_destiny
$final_banned_destiny
$final_spam_destiny
$final_bad_header_destiny
$warnvirussender
$warnspamsender #(not really used any more)
$warnbannedsender
$warnbadhsender
$warnvirusrecip
$warnbannedrecip
$warn_offsite
You can uncomment and set $spam_admin if you would like to get detailed diagnostic messages for each spam:
$spam_admin = "postmaster\@$mydomain";
Play with these until you have everything the way you prefer it (as far as notification and destination of all the garbage email goes). Experiment with them a little to get an idea of how they work. I recommend D_DISCARD for $final_virus_destiny in this day and age. Since many modern viruses and worms (like MYDOOM) fake the sender, D_BOUNCE just creates extra useless and confusing Internet traffic and it also clogs up your queues with mail that can't be delivered. It will still quarantine the message and notify the postmaster when a virus is found. If you would like all email containing a virus (or a banned attachment) to disappear into a black hole, set
$virus_quarantine_to = undef;.
******************************************************
Back up critical files:
Index******************************************************
Let's back up some of our hard work to a floppy. Grab a plain old High-Density 1.44MB floppy disk, and head on over to the spamfilter. Insert the floppy, and then head back to your Windows machine. Or just type this stuff in at the console if it's a long walk:
We may have to create a device if it does not exist:
test -e /dev/fd0u1722 || mknod /dev/fd0u1722 b 2 60
Create the mount point: mkdir /floppy
Format the floppy: fdformat /dev/fd0u1722 |
Create a file system on the floppy (we need one that accepts long file
names):mke2fs /dev/fd0u1722 |
Mount the floppy:mount /dev/fd0u1722 /floppy
Yes I know, working with floppies in *nix is a pain. |
And copy all these files to it: (Yes you can copy and paste this whole
section).
cp /etc/fstab /floppycp /etc/aliases /floppycp /etc/postfix/main.cf /floppycp /etc/postfix/master.cf /floppycp /etc/postfix/sender_access /floppycp /etc/postfix/transport /floppycp /etc/postfix/virtual /floppycp /etc/postfix/relay_recipients /floppycp /etc/postfix/body_checks /floppycp /etc/postfix/header_checks /floppycp /etc/amavis/amavisd.conf /floppycp /etc/spamassassin/local.cf /floppycp /var/lib/amavis/.spamassassin/user_prefs /floppycp /var/lib/amavis/.razor/razor-agent.conf /floppycp /etc/apt/sources.list /floppycp /etc/apt/preferences /floppycp /etc/clamav/clamd.conf /floppycp /etc/clamav/freshclam.conf /floppy
|
Wait a minute for the files to copy over, and then list the floppy disk's contents:
ls -l /floppyThis floppy will not be readable by a Windows machine. I can't repeat this enough: Always unmount the floppy before you remove it: I suggest you have the monitor on at the console so you can see the mess you make if you don't. umount /floppyRemove it, label it and store it in a safe place. |
If you wanted to restore the files at some point in the future:
mount /dev/fd0u1722 /floppy
cp /floppy/aliases /etc/aliases newaliases cp /floppy/main.cf /etc/postfix/main.cf cp /floppy/master.cf /etc/postfix/master.cf cp /floppy/sender_access /etc/postfix/sender_access postmap /etc/postfix/sender_access cp /floppy/transport /etc/postfix/transport postmap /etc/postfix/transport cp /floppy/virtual /etc/postfix/virtual postmap /etc/postfix/virtual cp /floppy/relay_recipients /etc/postfix/relay_recipients postmap /etc/postfix/relay_recipients cp /floppy/body_checks /etc/postfix/body_checks cp /floppy/header_checks /etc/postfix/header_checks cp /floppy/amavisd.conf /etc/amavis/amavisd.conf cp /floppy/user_prefs /var/lib/amavis/.spamassassin/user_prefs cp /floppy/local.cf /etc/spamassassin/local.cf cp /floppy/razor-agent.conf /var/lib/amavis/.razor/razor-agent.confcp /floppy/clamd.conf /etc/clamav/clamd.confcp /floppy/freshclam.conf /etc/clamav/freshclam.confcp /floppy/sources.list /etc/apt/sources.listcp /floppy/preferences /etc/apt/preferencesapt-get update
umount /floppy then shutdown and restart amavisd-new and Postfix as needed.
Note that we do not restore /etc/fstab, it is just a real good idea to have a copy of
to refer to. It describes our partition layout.
If you have an ftp server that you can upload files to, (maybe you got 10MB of web space with a dial up account you have) I will quickly describe backing up the files to that server. It's a good idea to have 2 terminal windows open so we can have ftp running in one, and the command line in the other.
Stop Postfix and amavisd-new so the bayes files are not written to during the backup:
postfix stop
amavisd-new stop
su amavis -c 'sa-learn --sync'
Start your ftp session (substituting your settings): ftp -p server.domain.tld
Enter your user name and password as requested, and then make a directory to place our files: ftp> mkdir sfa (only necessary the first time you do this)
Change to that directory: ftp> cd sfa
Then simply copy and paste this entire section:
ascii
|
get instead of put to restore a file.
For example: get clamd.conf /etc/clamav/clamd.conf
Don't get your servers mixed up if you have more than one!
******************************************************
Set up security reports:
Index******************************************************
We are first going to install
logcheck
apt-get install logcheck logcheck-database
Read the message that comes up. If you would like to change any settings: vi /etc/logcheck/logcheck.conf |
su -s /bin/bash -c "/usr/sbin/logcheck" logcheck at the command
prompt, wait a minute, and then check the
mailbox of the address you configured logcheck to send email. You should get a
message indicating the security violation. Logcheck looks for suspicious
activity and is scheduled to run once each hour. You will only get a message if
it finds something suspicious, but this is configurable. Don't set it to
"paranoid", there would be an entry for every email that passed through the
system.
Read more about logcheck by using less /usr/share/doc/logcheck/README.logcheck
If you get annoying repetitive emails from logcheck, try editing
/etc/logcheck/ignore.d.server/logcheck and insert
a regular expression of the text you wish logcheck to ignore. For example, I
insertedCRON.*: \(pam_unix\) session opened for userCRON.*: \(pam_unix\) session closed for user
|
You most likely will not want to be annoyed by every message amavisd-new Blocks,
so edit this file using vi or the WinSCP editor: vi /etc/logcheck/ignore.d.server/amavisd-new
At the bottom, you will find the text: amavis\[[0-9]+\]: +(\([-0-9]+\) +)?(SPAM|Not-Delivered|Passed|BANNED|INFECTED)
Change this to: amavis\[[0-9]+\]: +(\([-0-9]+\) +)?(Blocked|SPAM|Not-Delivered|Passed|BANNED|INFECTED)
Logcheck will also look for keywords (like "attack") contained in the /etc/logcheck/cracking.d/logcheck file. We can tell logcheck to ignore log entries that contain a hostname such as "attackingthedevil.co.uk" by creating a new file in the appropriate logcheck "ignore" directory and placing that text in it. We create a file because none exist at this point: echo "attackingthedevil.co.uk" >> /etc/logcheck/cracking.ignore.d/logcheck-postfix
If you removed the DCC log directory as I suggested, dccifd will log an error every time it is called. We want logcheck to ignore those log entries (this is a single command): echo "stat\(log directory /var/dcc/log\): No such file or directory" >> /etc/logcheck/ignore.d.server/dcc
While we are at it, there is another dcc message we don't care about: echo ": missing message body" >> /etc/logcheck/ignore.d.server/dcc
and also:
echo "detected 0 spam, ignored for 0, rejected for 0," >> /etc/logcheck/ignore.d.server/dcc
and a Postfix message I wish to suppress:
echo "dsn=2.7.0, status=sent \(254 2.7.0 Ok," >>/etc/logcheck/ignore.d.server/postfix
|
For more information about logcheck rules and patterns to include or ignore:
vi -R /usr/share/doc/logcheck-database/README.logcheck-database.gzand to debug logcheck:
su -s /bin/bash -c "/usr/sbin/logcheck -otd" logcheck
Subscribe to http://lists.debian.org/debian-security-announce/ and visit http://www.debian.org/security/
You will get an email from them every time any one of approximately 15,000 programs in the Debian catalog have been found to have a security flaw, along with some instructions. You will not have many of those programs installed on your system. The 'stable' version of a security related software update is released first. The 'testing' and/or 'unstable' versions may take a couple weeks to get released. To do a quick check whether you have a program on your system, try:
which [program name] or whatis [program name]
or dpkg -l [package name]
And to get a somewhat full list of packages that are installed on your system:
cdUse dpkg and grep to send the list of installed program
files to a file called "progs":dpkg -l '*' | grep '^i' > /root/progsThen use less to view the file:less /root/progsOr send it in an email to root: cat /root/progs | mail -s "sfa installed programs" root |
I like to be notified each morning that the mail queue on our system is empty (or nearly empty, depending on how busy our system is) because if it is not, we may have a problem. Note that if you do not use the "relayhost" parameter in /etc/postfix/main.cf, you should have items in the mailq most of the time. These will most likely be NDRs that cannot be delivered. If the queue is full of legitimate email, the most likely cause is our Exchange server is down. If you don't get a mailq email at all, it is likely the spamfilter is down. We will only display the last 10 lines of output from the mailq command.
crontab -eAnd insert: 30 7 * * * /usr/bin/mailq |/usr/bin/tail |/usr/bin/mail -s "mailq sfa" root
While we are editing crontab, let's also sync our CMOS clock once each day, and for the forgetful type (forgot that you are not supposed to run 'sa-learn' as root) we will make sure 'amavis' still owns the Bayes and AWL files: 10 10 * * * /sbin/hwclock --systohc30 17 * * * /bin/chown -R amavis:amavis /var/lib/amavis/.spamassassin
Save and exit |
In addition to
mailq (or as an alternate) you may wish to use qshape
vi /etc/cron.d/qshape-cronAnd insert (2 lines): PATH=/usr/sbin
31 7 * * * postfix /usr/sbin/qshape incoming active deferred 2>&1 |/usr/bin/mail -s "qshape sfa" root
Save and exit. Logcheck will now complain each day, so we need to shut it up: vi /etc/logcheck/ignore.d.server/postfix
and insert (1 line): ^\w{3} [ :0-9]{11} [._[:alnum:]-]+ postfix/pickup\[[0-9]+\]: [[:alnum:]]+: uid=[0-4]+ from=<postfix>
|
******************************************************
Set up intrusion detection:
Index******************************************************
Don't start this section unless you have a good hour to work on it. This part drove me nuts for a couple of days. I have used tripwire on my Red Hat boxes and felt it was pretty straightforward after I understood the concepts and the commands. When I tried it on Debian (it is available with apt-get) it simply refused to run (segmentation faults and such). I could not find the cause after Googling for a good hour. Any way, I gave up and looked for alternatives. I decided to give AIDE a try since it was also available as a Debian package. Be forewarned you may not enjoy this part. Be aware that your Linux box is an ideal platform to launch an attack on your entire network. So implementing this is highly recommended.
The concept with AIDE is to take a snapshot of some of the characteristics of critical files on our system, and then store that information on read-only media, a write protected floppy for example. Then run a program that compares the current state of those files on our system with the database, and report any differences. We should be able to tell if a hacker has compromised our system if the characteristics of the files on our system differ from the files in the database.
There are some challenges with doing this. Obviously, during normal use, some files change, like log files. So one challenge is to cut down on the unnecessary "chatter" that appears in the report. Adding and removing programs is considered normal use but this type of activity will generate alarms. When we are at the point the report has grown and is causing us to spend more time reading it than it should, we generate a new database and move it to the read only media. We have to be sure our system has not been compromised at the time we create a new database, so it would be wise to run a report before any new "trusted" software is installed, then another one after, then create a new database so we don't have to look at all that activity on every subsequent report.
When we install the program, it sets everything up on the hard drive, allows us to create the database right off the bat and it sets up a cron job to send us a report each day. This is fine, but completely insecure. A Hacker could replace the program, so we have to move it to the write protected media. We are going to use a write protected floppy disk, for convenience. The configuration file could be hacked, so we have to move it. And the same thing goes for the script that generates the report. These all have to be moved to write protected media. We can add an entry in crontab to run the report script off the floppy, and we will, but keep in mind that this can be hacked also, so the only way to be sure the report has not been tampered with is to run it from the command line off the write protected floppy.
Let's install aide:
For consistency, we are not using the most current version of AIDE:cd /usr/local/srcwget http://verchick.com/mecham/public_html/spam/aide_0.10-4_i386.debdpkg -i aide_0.10-4_i386.debThe installation will have three input screens, answer them as follows: Daily reports are mailed to root by default. [Ok] Initialize aide database? [No] Before AIDE can be used, you will have to initialize a database. [Ok] |
The installation should create these files:
/usr/bin/aide = the executable program file/etc/aide/aide.conf = the configuration file/etc/cron.daily/aide = a nice script that runs the report
Of course, other files and directories were created, but these are the only files we care about at this time.
We are going to move these files to /root/aide then uninstall the program.
Make a new directory, a convenient place to store the files we
want:mkdir /root/aide
Then copy the files, note that we will rename the report script "report": cp /usr/bin/aide /root/aidecp /etc/aide/aide.conf /root/aidecp /etc/cron.daily/aide /root/aide/report
Then we uninstall aide: apt-get --purge remove aideWe will need recreate the directory that AIDE uses for its logs: mkdir /var/log/aide
|
Let's work on the configuration file:
vi /root/aide/aide.confChange: database=file:/var/lib/aide/aide.dbTo: database=file:/floppy/aide.dbChange: database_out=file:/var/lib/aide/aide.db.newTo: database_out=file:/root/aide/aide.db.new
Under Custom Rules, edit Binlib, and remove: +m +c Edit Devices, and remove: +i +g +c under # Kernel, change:
=/boot$ Binlib
to:
/boot Binlib
Comment out the entire section under # Log Files
Under # Check crontabs add:/etc/cron.d DatabasesSave and exit. |
http://www.cs.tut.fi/~rammer/aide/manual.html
Let's edit the report script:
vi /root/aide/reportChange: PATH="/bin:/usr/bin"To: PATH="/floppy:/bin"Change: CONFFILE="/etc/aide/aide.conf"To: CONFFILE="/floppy/aide.conf"Change: [ -f /usr/bin/aide ] || exit 0To: [ -f /floppy/aide ] || exit 0If you would like to send the report to someone other than root, optionally change: MAILTO="${MAILTO:-root}"To something like: MAILTO="${MAILTO:-someuser@example.com}"Change: DATABASE="${DATABASE:-/var/lib/aide/aide.db}"To: DATABASE="${DATABASE:-/floppy/aide.db}"Change: COMMAND="${COMMAND:-check}"To: COMMAND="${COMMAND:-update}"
Change: aide $AIDEARGS --$COMMAND >"$LOGDIR/$LOGFILE" 2>"$ERRORTMP"To: aide $AIDEARGS --config=/floppy/aide.conf --$COMMAND >"$LOGDIR/$LOGFILE"
2>"$ERRORTMP"Save and exit. |
If you have a floppy in the drive, unmount it (
umount /floppy)
then remove it.
Insert a floppy in the drive. We are going to format it, create
an ext2 file system on it, mount it, and then copy the AIDE files to it.
Make sure we have a mount point:
mkdir /floppy
Run these one at a time and wait for each command to finish: fdformat /dev/fd0u1722
mke2fs /dev/fd0u1722
fsck /dev/fd0u1722
mount /dev/fd0u1722 /floppy
|
Then copy the files (you can copy and paste this entire section):
cp /root/aide/aide /floppy |
Wait for the files to copy over (30 seconds).
Now we can run the routine that creates the first database. We only need to run this once:
/floppy/aide --config=/floppy/aide.conf --init |
This creates aide.db.new in the /root/aide directory, this is the initial database and must be copied to the floppy as aide.db.
cp /root/aide/aide.db.new /floppy/aide.db |
Wait long enough (20 seconds) for the file to copy over, and then run the report:
/floppy/report |
cat /var/log/aide/error.log
Now you have to write protect the floppy disk. This is the most important part.
umount /floppyThen pull out the disk and push up the write protect tab. Reinsert the floppy, and mount it again: mount /dev/fd0u1722 /floppy
This floppy should remain in the drive all the time. (Unless we need to use the drive for a moment) |
Each time you run the report, it reads the database /floppy/aide.db but it also creates a new aide.db.new database. So why do we care about the aide.db.new? It's a matter of convenience. If the report is getting kind of long, and there appears to be nothing in it you would not expect; the aide.db.new database reflects the current state of the system and could be used to replace aide.db. You replace your aide.db on the floppy with aide.db.new, just as you did when you first initialized the system. If you run the report again, it should be clean, unless files have changed since it was created. You need to understand the risks of relying on the quality of the aide.db.new database if you use the copy that was created during the cron job. The cron job could have been hacked. A hacker could manipulate the file. Only rely on the aide.db.new database if you run
/floppy/report from the command line and
you inspect the report just prior to copying it over to the floppy. If you
left the write protection off the floppy drive for any length of time, it's possible you
can no longer rely on the database. Be paranoid.
So here's the gist of maintenance:
- Run the report.
- Read the report.
- If it looks normal, and it has not grown to the point it is annoying, you're done for today.
- If the report lists any critical system files you know you did not change
or update, or you notice other suspicious activity,
I hope you have a cloned hard drive ready to swap out because you may have been hacked. Study the report carefully. - If everything looks normal and you want it cleaned up, unmount the floppy,
flip the write protect tab off, mount the floppy,
copy /root/aide/aide.db.new to /floppy/aide.db, unmount the floppy, flip the write protect tab on, mount the floppy, done.
You could put the command /floppy/report in your crontab (we will do so), and have it run each day just before you normally open your email, but a hacker could send forged reports for weeks before you knew there was a problem. Be aware of the danger. For a trustworthy report simply run the report from a command line every day, few days, week, few weeks, month, whatever your level of paranoia permits.
crontab -eAnd insert: 25 7 * * * /floppy/reportSave and exit |
Ok, you should make a backup copy of the floppy, because floppies are not reliable:
Change to our home directory:cdUnmount the floppy, but leave it in the drive: umount /floppyCreate an image of the floppy and store it to a file: dd if=/dev/fd0u1722 of=floppy.imgdd stands for `Copy and Convert' and was renamed to `dd` only because `cc' was reserved for the C compiler. if= input file, of= output file Remove the source disk, insert the destination disk: First, format the new floppy: fdformat /dev/fd0u1722
Then copy the image to the new floppy: dd if=floppy.img of=/dev/fd0u1722
Simply use the new disk now. Turn write protect on, then mount the floppy: mount /dev/fd0u1722 /floppy |
We will create a special startup script called /etc/init.d/startflop that runs when the system boots up. We will use this script to mount the floppy automatically when we boot up. There are other ways to accomplish this task, but this one works best. Depending on your hardware, you may need to set your BIOS to boot from the hard drive first to avoid it from attempting to boot from the floppy.
vi /etc/init.d/startflopAnd insert the following text just as it is listed here (Hint: use the [tab] key, not a bunch of spaces) Note that this will not paste well using the right-click method. Try [Shift]+[Insert] to paste: |
#! /bin/sh # # very simple startup script case "$1" in start) /usr/bin/test -e /dev/fd0u1722 || /bin/mknod /dev/fd0u1722 b 2 60 /bin/chmod 660 /dev/fd0u1722 /bin/chown root:floppy /dev/fd0u1722 /bin/mount /dev/fd0u1722 /floppy ;; stop) /bin/umount /floppy ;; esac exit 0 |
Save and exit the file, then make the file executable:chmod +x /etc/init.d/startflopNow use this command to enable it and prioritize it: update-rc.d startflop defaults 80Now make a symbolic link to it, I will explain why in a moment: ln -s /etc/init.d/startflop /usr/bin/floppy |
The symbolic link will make our life easier. Now, to mount the floppy simply run:
floppy start
and to unmount the floppy simply run:
floppy stop
Note that this can only be used to mount floppies that we formatted with "fdformat /dev/fd0u1722"
See how little free space we have on the floppy disk:
df /floppy |
See http://www.oreilly.com/catalog/debian/chapter/book/ch04_04.html for help with mounting devices.
******************************************************
CPAN, Mailgrep, Pflogsumm and trim_whitelist:
Index******************************************************
We are going to install a few utilities that will be of value to us. One is mailgrep.pl that makes it easier to find information in the mail.log file and another is pflogsumm.pl that produces statistical reports. We will also install a script to maintain the auto-whitelist file. You could install pflogsumm via apt-get but doing it this way familiarizes us with CPAN.
We need to install a couple of programs using CPAN; CPAN is a system created to make it easier to install Perl modules.
See http://www.cpan.org/ for more information about CPAN.
The first time you start up CPAN, it will ask you to configure it. We want to answer "yes" to configure it. We will accept all the default prompts except we need to use http mirrors and not ftp mirrors. If you have a proxy server in front of your spamfilter that requires user names or passwords or special settings, I can't help you with that. You'll just have to look elsewhere for answers. One more note: if you want to start the configuration process over again, at the
cpan> prompt, enter o conf
init
We need to set our system to use passive mode for ftp transfers:
vi /etc/libnet.cfgAnd change: ftp_ext_passive => 0,to: ftp_ext_passive => 1, |
Start a CPAN session:perl -MCPAN -e shellAccept the default of "yes" at the: Are you ready for manual configuration? [yes] Accept all the defaults, eventually you will need to (and this may take some time to come up): Select your continentSelect your countryAfter you have selected these, hit [space][enter] a few times until you see some http servers. Do not pick any ftp servers. Select a couple of the http servers. Hit [enter] to get back to the cpan> prompt. |
Now let's install a couple of modules we need:
At the cpan> prompt type these commands in exactly:install File::MMagicIf you are asked any questions, accept the defaults. Back at the cpan> prompt: install Date::CalcIf you are asked any questions, accept the defaults. When it finishes, exit cpan with: q |
While not required, a nice tool to have is "mailgrep.pl" which parses certain data out of the mail logs instead of having to scroll through them, or do the grep commands. Craig Sanders wrote this little gem (http://taz.net.au/postfix). To set this up, download the openlogfile (a perl function to open log files) and mailgrep (a perl script to help search the maillog) utilities, rename them, put them in /usr/bin, and make mailgrep.pl executable:
cd /usr/bin
|
To learn how to use this, just type the command:
mailgrep.pl
To search for all mail log entries dealing with mail to or from "someuser@somedomain.com", we would use:
mailgrep.pl -s someuser@somedomain.com /var/log/mail.log or
mailgrep.pl -s someuser@somedomain.com /var/log/mail.log.0 to search slightly older logs.
To see what mailgrep.pl does for you, compare the output of the above to:
grep -i someuser@somedomain.com /var/log/mail.log
This section will set up an automated preparation of a very nicely laid out report of email activity, and email this report to us once per day, with the previous day's mail statistics. We need the pflogsumm.pl utility:
|
Go to:
http://jimsun.linxnet.com/postfix_contrib.html And by looking at the link to the program, edit these next lines if necessary to reflect the latest production version:
cd /usr/local/src
|
Let's set up a cron.daily script that prepares and mails us the pflogsumm report. When I first wrote this HOWTO, the mail.log file rotated once each week. Now it seems to rotate sometimes once a day, sometimes twice a day and sometimes every couple days. I used to be able to create a weekly report when the mail.log files were rotated on a predictable basis but now I have to combine several log files just to be insured I have the data I need to report on the previous day's activity.
cd /etc/cron.dailywget http://verchick.com/mecham/public_html/spam/pflogsummchmod +x pflogsummYou may need to edit the script and change the hostname just after the word "DAILY" to reflect your system's hostname. You may also want to change to whom the report gets mailed: vi /etc/cron.daily/pflogsumm |
There is one major problem with the report. Any email that gets sent to amavisd-new, which includes most mail that isn't rejected at the front door, also comes back from amavisd-new. This means Postfix sees the email twice. So the report lists them twice. Sorry.
I suggest you read the pflogsumm.pl faq; your version number of pflogsumm may be different:
less /usr/local/src/pflogsumm-1.1.0/pflogsumm-faq.txt
|
FYI:
/etc/crontab controls when the scripts in /etc/cron.daily
/etc/cron.weekly and /etc/cron.monthly run.
One of the scripts in /etc/cron.weekly
is sysklogd. sysklogd reads its configuration from /etc/syslog.conf. You can see
what logs should get rotated weekly by sysklogd by using the command
syslogd-listfiles --weekly
You will notice that if you use SpamAssassin's auto-whitelist feature, over time the
/var/lib/amavis/.spamassassin/auto-whitelist
file will grow to a rather large size. The SpamAssassin source code comes with a utility
called check_whitelist that can be used to trim the size down.
Run perldoc check_whitelist to see how it is used. There is a
modified version of check_whitelist called trim_whitelist we are going to use to
reduce the size of the auto-whitelist file. This utility is explained here:
http://article.gmane.org/gmane.mail.spam.spamassassin.general/59651.
Download trim_whitelist:cd /usr/sbinwget http://www.deepnet.cx/~kdeugau/spamtools/trim_whitelist
Make it executable: chmod +x /usr/sbin/trim_whitelist
List our SpamAssassin directory: ls -l /var/lib/amavis/.spamassassin
Run the program in order to test it: su amavis -c '/usr/sbin/trim_whitelist'
List our SpamAssassin directory again: ls -l /var/lib/amavis/.spamassassin
If the test is successful, the program will have created a new file: auto-whitelist-old
Now we will create a cron job so trim_whitelist will run once each week: cd /etc/cron.weeklywget http://verchick.com/mecham/public_html/debian/trim_whitelist_weeklychmod +x trim_whitelist_weeklycd
|
******************************************************
Whitelisting, Blacklisting, Tweaking:
Index******************************************************
Postfix, Amavis and SpamAssassin all have methods to do whitelisting and blacklisting. Postfix has a number of methods to reject mail before it enters the system, we configured some and I provided links to others during the Postfix Anti Spam configuration. We also discussed
/etc/postfix/sender_access
that can be used to blacklist senders. We created
/etc/postfix/header_checks and /etc/postfix/body_checks
that can be used for content filtering and I provided links to some examples.
The files themselves also provide examples. When we were editing
/etc/amavis/amavisd.conf you noticed sections that dealt with
whitelisting and blacklisting. It is recommended you do "soft" whitelisting
and blacklisting by adding entries to the @score_sender_maps section of amavisd.conf.
Add your entries in the same section that 'nobody@cert.org' => -3.0, is
listed. Negative scores will be subtracted from the overall spam score, and positive
scores will be added.
Note that Postfix and Amavis will need to be shutdown and restarted (or reloaded) after making changes to configuration files:
postfix reload will cause Postfix to reload its configuration files
/etc/init.d/postfix restart will shutdown and restart Postfix, and so will:
postfix stop postfix start /etc/init.d/amavis stop /etc/init.d/amavis start is the best way to handle Amavis, but this works just as well:
amavisd-new stop amavisd-new start
You can stop and restart amavisd-new while Postfix is running. Delivery to amavisd-new will fail, so Postfix will queue the mail in the deferred queue. If amavisd-new has only been down for a moment, Postfix will try to send the mail again in a few minutes. If you are impatient, or amavisd-new has been down for a longer period, you may wish to use 'postfix flush' to force Postfix to attempt delivery of the mail currently in the deferred queue. If you have made changes to the way postfix communicates with amavisd-new (for example you comment out content_filter in main.cf to bypass amavisd-new) you may need to requeue the mail with
postsuper -r ALL.
Configuring and tweaking SpamAssassin is an art in itself. There is plenty of information on the Internet to help guide you through this. I will simply supply a few tweaking links for you to explore. Don't forget that it is possible to write custom rules with negative numbers that can be used to lower the score. Keep in mind that many custom rules once only available as add-ons are now integrated into SpamAssassin version 3, so do some research to find out if you are duplicating rules.
Many people use by Rules Du Jour, but remember, if you use a lot of custom rules, you can use more than 100MB RAM for each amavisd process (and create a major load on the system). Because some of these rule sets are updated frequently it increases the possibility that a rule set may have an error in it. If this happens it could halt your server. NEVER use blacklist.cf, blacklist-uri.cf or bigevil.cf. I suggest you go slow, add one at a time, then wait a day or so and check your memory usage: http://www.exit0.us/index.php?pagename=RulesDuJour
As far as custom rules go, I personally use http://www.rulesemporium.com/rules/mangled.cf
(Assuming you are using SA version 3.1.x) I also download a set of the safest SARE rules: http://marc.theaimsgroup.com/?l=spamassassin-users&m=115637139728022
cd /etc/spamassassin
wget http://saupdates.openprotect.com/pub.gpg
sa-update --import pub.gpg
sa-update --gpgkey D1C035168C1EBC08464946DA258CDB3ABDE9DC10 --channel saupdates.openprotect.com
spamassassin --lint
and if everything looks OK:
/etc/init.d/amavis restart
Personally I use sa-update, and the OpenProtect SARE rules shown in the link above. Once these are properly configured, you could use something like my script that downloads updates from both places, and if updates are downloaded from one or he other, amavisd-new is restarted:
cd /usr/sbin
wget http://verchick.com/mecham/public_html/spam/sa-update.sh
chmod +x sa-update.sh
You would place /usr/sbin/sa-update.sh in your crontab (replacing the existing sa-update command).
And here is a note on how to use sa-update instead of RulesDuJour: http://marc.theaimsgroup.com/?l=spamassassin-users&m=115545719618240
Remember, adding custom rule sets will require additional RAM.
Use
top to see what's using memory;
then change the sort order with > and <
SpamAssassin Rules Emporium (SARE):
http://www.rulesemporium.com/
To learn more about configuring SpamAssassin, run this at the command prompt:
perldoc Mail::SpamAssassin::Conf
Of course visit the SpamAssassin site:
http://spamassassin.apache.org/
Visit the SpamAssassin Wiki for more ideas:
http://wiki.apache.org/spamassassin/FrontPage
and the Exit0 SpamAssassin Wiki
http://www.exit0.us/
This is an example of a /etc/spamassassin/local.cf file:
bayes_path /var/lib/amavis/.spamassassin/bayes auto_whitelist_path /var/lib/amavis/.spamassassin/auto-whitelist lock_method flock # # We need stuff from these senders, and they tend to get marked as spam. # We want to whitelist our close business partners. # We subscribe to industry specific newsletters and whitelist them also. # Later we manually feed these to Bayes as ham. whitelist_from spambin@example.com whitelist_from *@generalmotors.com whitelist_from *.usanewstoday.com # # We need stuff from autonetamerica and the Lottery and it always gets marked as spam. # So we will create custom rules that let these particular subject lines reduce the score. header AUTONETAMERICA Subject =~ /Auto Net America/ score AUTONETAMERICA -5.000 header YOURLOTTERY Subject =~ /Your Lottery Results!/ score YOURLOTTERY -5.000 # # We change the scores on a few standard tests - these are just examples score RAZOR2_CF_RANGE_51_100 0.500 score URIBL_WS_SURBL 2.000 score URIBL_PH_SURBL 2.500 score RCVD_IN_SORBS_HTTP 1.000 score RCVD_IN_SBL 1.000 score RCVD_IN_NJABL_PROXY 1.000 score RCVD_IN_SORBS_MISC 0.500 score RCVD_IN_BL_SPAMCOP_NET 2.000 score RCVD_IN_NJABL_SPAM 2.200 # # use_auto_whitelist 0 # uncomment to disable auto-whitelist - a number of people recommend NOT using auto-whitelist.
Make sure you run
spamassassin --lint after adding any new rules.
Normally, SpamAssassin is set up as a multi user system and users are able to tweak their own personal settings by editing their individual user_prefs file. In our case the only users that use SpamAssassin are amavis and root. The
/etc/spamassassin/local.cf file is used to
configure SpamAssassin site wide but in our case,
editing /var/lib/amavis/.spamassassin/user_prefs
would have the same effect. However, certain global SpamAssassin settings will
have no effect if placed in user_prefs, therefore you only need to maintain local.cf.
|
If image spam is a problem for you, please see another one of my documents that attempts
to address that issue. This type of spam is particularly dangerous because it tries
very hard to corrupt your Bayes database: http://verchick.com/mecham/public_html/spam/image_spam2.html |
******************************************************
Create Rescue Floppy:
Index******************************************************
Because of the size of the kernel, you cannot make rescue floppies from this version of Debian. This rescue floppy may allow you access to the files on your hard drive if you cannot boot from the hard drive. This disk may not work if you have modified /etc/init.d/sysklogd as you may have done if you installed bind9. I was however able to repair LILO or GRUB on a system using the tomsrtbt disk described in the next section. If neither this rescue floppy nor a tomsrtbt disk allows you access to the system, a third option would be to create an Etch CD (even the business card is fine) and use it as a rescue CD. http://www.debian.org/devel/debian-installer/index Boot up using the Etch CD, answer the first few prompts (Language, Country, Keyboard), let it discover devices (but go no further!), then use [Alt]+F2 to open a console. At this point you would jump ahead and read in the tomsrtbt section how to mount your hard drive and repair GRUB or LILO. If you are using a 2.6.15 or greater kernel, then using the Etch CD may be your best (or only) option, but if you are using an older kernel, then you should try either the rescue floppy or tomsrtbt disk first.
We are going to download an "image file" into our debian directory that will be used to create the floppy.
|
If you have not already done so, browse to:
http://uranus.it.swin.edu.au/~jn/linux/rawwrite.htm |
| Browse to: http://mirrors.kernel.org/debian/dists/woody/main/disks-i386/3.0.23-2002-05-21/images-1.44/bf2.4/ |
We will need two more files to make this disk work. I want you to:
| Right-click these next 2 links and "Save Target as" or "Save Link
Target as" and save them to the debian folder. http://verchick.com/mecham/public_html/debian-test1/message.txt http://verchick.com/mecham/public_html/debian-test1/syslinux.cfg |
The rescue floppy we will make is a SYSLINUX disk. Read more if you like at http://syslinux.zytor.com/. This rescue floppy, the way we will configure it, assumes a few things. It assumes you are using an IDE drive and it assumes your root directory is mounted to /dev/hda1. It also assumes you are using ext2 or ext3 file system; I'm not sure about other file systems. It assumes you are not using RAID. This is possibly how we are going to configure our system, so this rescue disk is semi-custom. If you have a SCSI disk, edit the syslinux.cfg file and change /dev/hda1 to /dev/sda1. If root is mounted somewhere other than partition 1, change that also (/dev/hda5 etc.). You can also edit the message.txt file to reflect the changes you made to syslinux.cfg. Run 'df' for a quick overview of your file system.
You will need a standard blank 1.44MB IBM formatted floppy inserted in the floppy drive of your Windows machine. Now simply start up rawwritewin.exe, browse for rescue.bin (All files *.*) and Write the disk. Once you have edited the syslinux.cfg file to reflect your device name and location of the root (/) mount point, copy it to the floppy, overwriting the copy currently on the disk. Also edit message.txt if required and copy it to the floppy. Done.
As mentioned before, if you are able to boot up using this disk, and you use LILO, you can simply run 'lilo' to repair the boot sector. If you use GRUB, then run this command (edit as needed to reflect the type of disk you are using):
/sbin/grub-install /dev/hda
[Ctrl][Alt][Del]
******************************************************
Make a tomsrtbt disk:
Index******************************************************
I promised I would tell you how to make a tomsrtbt disk. We can boot to this disk, and then run dd to duplicate the hard drive as explained at the beginning of this document. We can also use this disk to repair LILO or GRUB after a normal type of clone using Ghost.
umount /floppy
Now it's safe to remove the floppy currently in the drive. Please check http://ftp.sunsite.utk.edu/ftp/pub/mini-linux/tomsrtbt/ for version changes and edit if necessary: Insert a blank floppy. Then: cd /usr/local/src
wget http://ftp.sunsite.utk.edu/ftp/pub/mini-linux/tomsrtbt/tomsrtbt-2.0.103.tar.gz
tar xzvf tomsrtbt-2.0.103.tar.gz
rm toms*.gz
cd tomsrtbt-2.0.103
./install.s
|
Remove and label the disk when the process is finished, then reinsert our AIDE floppy.
mount /dev/fd0u1722 /floppycd |
If and when the time comes we actually use this floppy of course we will have to set the BIOS to boot from the floppy drive.
Here is how I used a tomsrtbt disk to repair LILO after a disk clone using Ghost 2003. You MUST know in advance which partition is your root partition. You should keep a copy of the text in /etc/fstab to refer to. In this example we will assume the root partition was mounted on /dev/hda6, a separate /boot partition was mounted on /dev/hda1 and you are repairing LILO. If you use other mount points (like /var for example) (depending on what you are trying to repair) you may need to mount those too.
To repair LILO, boot up using the tomsrtbt disk.
mkdir /hardroot
mount /dev/hda6 /hardroot
chroot /hardroot
(If you have a separate boot partition, this would need to be mounted too):
mount /dev/hda1 /boot
cp /etc/lilo.conf /etc/lilo.conf-original
/sbin/lilo
Use [Ctrl][Alt][Del] to reboot. I got a bunch of errors when I ran 'lilo', but it booted up just fine. Once it reboots you should run:
shutdown -r -F now
To repair GRUB, run this command (edit as needed to reflect the type of disk you are using):
/sbin/grub-install /dev/hda
[Ctrl][Alt][Del]
******************************************************
Links, FAQs and such:
Index******************************************************
debian http://www.debian.org
postfix http://www.postfix.org
amavisd-new http://www.ijs.si/software/amavisd/
SpamAssassin http://www.spamassassin.org
Razor http://razor.sourceforge.net
DCC http://www.rhyolite.com/anti-spam/dcc/
Pyzor http://pyzor.sourceforge.net
ClamAV http://www.clamav.net
Similar Documents (for other distributions):
http://www.geekcomix.com/cgi-bin/classnotes/wiki.pl?Setting_Up_An_Anti-SPAM_Gateway
http://www.flakshack.com/anti-spam/
http://www.freespamfilter.org/
http://ezine.daemonnews.org/200309/postfix-spamassassin.html
http://freshmeat.net/articles/view/857/
http://www.gentoo.org/doc/en/mailfilter-guide.xml?style=printable
http://www.novell.com/coolsolutions/feature/16093.html
Other Debian documents that also include local mail delivery:
http://workaround.org/articles/ispmail-sarge/
http://www.fatofthelan.com/articles/articles.php?pid=22
http://www.falkotimme.com/howtos/perfect_setup_debian_sarge/index.php
Would you like a web based interface for per-user settings and quarantine control? Don't install this on a production machine until you are comfortable with it and keep in mind that the quarantined email is stored in a MySQL database on this machine which means you need a disk partition large enough to hold it. This software is not without issues and installing and configuring Maia Mailguard is not trivial so I suggest you read through the documentation and mailing list archives.
http://www.renaissoft.com/projects/maia/index.php
http://www.renaissoft.com/pipermail/maia-users/
http://www.linuxjournal.com/article/7427
If you are interested, this will help you get it installed: http://verchick.com/mecham/public_html/spam/debian-maia.html
Email testing tools:
http://www.declude.com/Articles.asp?ID=100
http://zmailer.org/mxverify.html
http://www.gfi.com/emailsecuritytest/
http://dnsreport.com/ and http://www.dnsstuff.com/
http://www.eicar.org/anti_virus_test_file.htm
http://www.webmail.us/testvirus
http://rbls.org/
http://spamlinks.net/tools-relay.htm#web
apt-get and dpkg:
http://oat.tao.ca/node/view/141
http://www.knoppix.net/forum/viewtopic.php?t=2638
http://jaqque.sbih.org/kplug/apt-pinning.html
http://www.debian.org/doc/manuals/apt-howto/
http://newbiedoc.sourceforge.net/tutorials/apt-get-intro/info.html.en
http://www.oreilly.com/catalog/linuxnut4/chapter/ch05.pdf
neat way to upgrade only 'stable' packages:
http://www.freespamfilter.org/forum/viewtopic.php?t=265
iptables:
http://iptables-tutorial.frozentux.net/iptables-tutorial.html
http://www.cites.uiuc.edu/wsg/talks/iptables/iptables-rh8.rules.txt
http://www.hackinglinuxexposed.com/articles/20030703.html
http://www.justlinux.com/nhf/Security/IPtables_Basics.html
http://huizen.dto.tudelft.nl/devries/security/iptables_example.html
http://morizot.net/firewall/gen/
DNS based spam databases (RBLs):
http://www.declude.com/junkmail/support/ip4r.htm
UTF-8
http://www.lugod.org/mailinglists/archives/vox-tech/2003-06/msg00128.html
Postfix:
http://www.postfix.org/BASIC_CONFIGURATION_README.html
http://www.postfix.org/STANDARD_CONFIGURATION_README.html
http://www.stahl.bau.tu-bs.de/~hildeb/postfix/
http://www.postfix-book.com/debugging.html
http://www.informatik.uni-bonn.de/pub/software/postfix/addon.html
http://www.metaconsultancy.com/whitepapers/smtp.htm
http://mywebpages.comcast.net/mkettler/sa/SA-rules-howto.txt
http://www.mines.edu/academic/computer/spamuserprefs.shtml
http://www.securitysage.com/antispam/
http://www.postfix.org/uce.html
http://jimsun.linxnet.com/misc/postfix-anti-UCE.txt
http://www.geekounet.org/filters/header_checks
http://www.fredshack.com/docs/postfix.html
Security:
http://www.debian.org/doc/manuals/securing-debian-howto/
http://linuxgazette.net/issue89/gonzales.html
http://www.acm.org/crossroads/xrds6-1/linuxsec.html
http://www.tldp.org/HOWTO/Security-HOWTO/index.html
http://www.freefire.org/lib/hardening.en.php3
Linux:
http://linux.about.com/library/bl/open/newbie/blnewbie_toc.htm
http://loll.sourceforge.net/linux/links/index.html
http://www.fifi.org/cgi-bin/man2html
http://wireless.ictp.trieste.it/school_2002/lectures/fonda/MoreLinux/rute/node1.html
Cron:
http://www.unixgeeks.org/security/newbie/unix/cron-1.html
http://www.pantz.org/os/linux/programs/cron.shtml
AIDE:
http://linuxgazette.net/issue75/maiorano.html
http://www.gnu.org/directory/aide.html
Debian:
http://articles.linmagau.org/modules.php?op=modload&name=Sections&file=index&req=viewarticle&artid=158&page=1
http://www.debianplanet.org/index.php?or=7
http://newbiedoc.sourceforge.net/
http://www.debian.org/distrib/packages
http://www.desktop-linux.net/debian-rclocal.htm
http://www.backports.org
http://www.falkotimme.com/howtos/index.php
Amavisd-new and SpamAssassin:
http://wiki.apache.org/spamassassin/FrontPage
http://www.exit0.us/
http://www.rulesemporium.com/index.html
http://wiki.apache.org/spamassassin/CustomRulesets
http://www.ijs.si/software/amavisd/#faq-spam
http://sourceforge.net/mailarchive/forum.php?forum=amavis-user
http://news.gmane.org/gmane.mail.spam.spamassassin.general
http://www.surbl.org/
Spam Links:
http://spamlinks.net/
Regular Expressions (Regex):
http://www.regular-expressions.info
http://www.silverstones.com/thebat/Regex.html#intro
http://gmckinney.info/resources/regex.pdf
http://www.contactor.se/~dast/mail2sms/regex.shtml
http://www.quanetic.com/regex.php
http://www.koralsoft.com/regextester/
http://www.renatomancuso.com/software/pcreworkbench/pcreworkbench.htm
Disk Cloning:
http://service1.symantec.com/SUPPORT/ghost.nsf/docid/2001111413481325?Open&src=&docid=19
http://www.rajeevnet.com/hacks_hints/os_clone/os_cloning.html
http://service1.symantec.com/SUPPORT/ghost.nsf/docid/2000042113083825?Open&src=&docid=1999021909463125
Populating relay_recipient_maps:
http://www2.origogeneris.com:4000/relay_recipients.html
http://www-personal.umich.edu/~malth/gaptuning/postfix/
http://www.unixwiz.net/techtips/postfix-exchange-users.html
http://postfix.state-of-mind.de/patrick.koetter/mailrelay/
Using eDirectory to provide real-time userid pass/fail to Postfix using OpenLDAP:
http://www.novell.com/coolsolutions/gwmag/features/trenches/tr_postfix_gw.html
Configuration files quick reference:
/etc/aliases/etc/postfix/main.cf/etc/postfix/master.cf/etc/postfix/sender_access/etc/postfix/transport/etc/postfix/virtual/etc/postfix/relay_recipients/etc/postfix/body_checks/etc/postfix/header_checks/etc/amavis/amavisd.conf/var/lib/amavis/.spamassassin/user_prefs/etc/spamassassin/local.cf/var/lib/amavis/.razor/razor-agent.conf/etc/clamav/clamd.conf/etc/apt/preferences/etc/apt/sources.list
My little fixit script: Copy and paste entire portion.
postmap /etc/postfix/sender_access |
A few unmentioned commands and FAQs:
How do I tell how much space is left on the drive?
Answer:
df
How do I tell how my hard drive is partitioned?
fdisk -l /dev/hda or /dev/sda
How do I delete just one email that is in the queue?
Answer: first run "mailq" and note the ID number. Then run
postsuper -d <ID number>
I'm still in the testing stages of this box and I have a bunch of test messages in the queue that I want to delete, how do I do this?
Answer:
postsuper -d ALL deferred
This will delete messages in the deferred queue.
I would like to change the IP address of this box, how do I do this?.
Stop Postfix and amavisd-new.
Look in these files, and edit as needed:
/etc/network/interfaces
/etc/networks
/etc/hosts
/etc/resolv.conf
/etc/postfix/main.cf
possibly /etc/postfix/sender_access
Run "postmap /etc/postfix/sender_access"
possibly /etc/postfix/transport
Run "postmap /etc/postfix/transport"
possibly /etc/spamassassin/local.cf
Modify your firewall settings, and send the iptables commands.
reboot
Modify other computers' hosts file(s) as needed.
Change PuTTY to the new address
If you find more places to change, let me know,
I want to insure postmaster@example.com receives mail even if it is spam, how do I do that?
Set:
@spam_lovers_maps = ( ['postmaster@example.com', 'abuse@example.com'], );
How do I enable per-user settings using LDAP or SQL?
Sorry, other than using the Maia Mailguard software listed above, I don't have any experience.
I want to have my Exchange box send its mail out through the spamfilter, so I can further isolate it from the Internet.
See these posts: http://www.freespamfilter.org/forum/viewtopic.php?t=82
http://marc.theaimsgroup.com/?l=amavis-user&m=113415019700881&w=2
The post above also explains how to use altermime to attach a disclaimer to outgoing mail (mail that is sent from the Exchange server).
Some final comments:
Spam traffic is increasing dramatically. About 90% of the email that passes through my system is spam. Setting this system up won't be the end of your anti-spam project, I'm afraid. You'll catch a LOT of spam with this, but some will keep getting through, and some good email will get blocked. You'll learn and tweak things as you go forward.
I hope this doc is useful to you. Feel free to contact me, mr88talent at yahoo dot com, with comments and suggestions or corrections. This website was created for my own personal use and entertainment.
Everyone is welcome to copy all or any portion of this document for their own use and/or to share with others pretty much as they see fit, provided it is done so without malicious intent and you do not claim that you wrote this document.
******************************************************
Disclaimer:
Index******************************************************
This website was created for the author's personal use and entertainment. There is absolutely no warranty. Use entirely at your own risk. Any information contained herein is freely available elsewhere and simply reinterpreted, or more likely misinterpreted, and cannot be assumed to be accurate. There are mistakes in this website and there may or may not be any effort to correct those mistakes in the future. The author accepts no responsibility for any loss or damage caused by the use, lack of use, or misuse, of information contained in this website. Where links are provided to other websites, the author accepts no responsibility and shall not be liable, either directly or indirectly for the content, legality, accuracy, reliability, suitability, quality or decency of content, information, product, advice or services provided by and contained in those sites. Downloading any information from the Internet is done at your own risk, and the risk can be substantial. You knew that, right? All trademarks are the property of their respective owners.
Buy your next car at: Salt Lake City As we say in Utah: pre-shayt-cha (I appreciate you) |