2015/05/18 – Updated for cPanel 11.50
Suggestions can be sent to ‘paul.trost’ at ‘trostfamily.org’
This article will show you how to replicate most of the functionality of a Barracuda Spam Firewall on you cPanel server. The BSF has 12 layers that messages would be process through as illustrated in the graphic below. I’ll show you how to replicate most of these steps using customizations to the Exim and Spam Assassin. This guide assumes you have a basic working knowledge of cPanel, Linux, Exim, and Spam Assassin.
I’m going to outline my steps slightly differently than on the Barracuda so you get a better idea of what steps is configured where.
- Step 1 – Network Denial of Service Protection
- Step 2 – Exim Rate Control
- Step 3 – IP Reputation Analysis
- Step 4 – Recipient Verification
- Step 5 – Sender Verification
- Step 6 – Content (Virus and Spam) Scanning
- Step 7 – Fingerprint Analysis
- Step 8 – Intent Analysis
- Step 9 – Image Analysis
- Step 10 – Bayesian Analysis
- Step 11 – Rule-Based Scoring
- Step 12 – Custom Policy
The text in bold type at the start of each step comes from the Barracuda Spam Firewall Email Filtering Whitepaper
Step 1: Network Denial of Service Protection
Built on a hardened and secure operating system, the Barracuda Spam Firewall receives email on behalf of the organization, insulating the organization’s email server from receiving direct Internet connections and the associated threats.
As filtering is being done on your cPanel server there is probably not going to be any front end device on your network to do DOS prevention. If not then I would recommend trying out the script at: http://deflate.medialayer.com/
Step 2: Rate Control
Automated spam software can be used to send large amounts of email to a single email server. To protect the email infrastructure from these flood-based attacks, the Barracuda Spam Firewall counts the number of incoming connections from a particular IP address and throttles the connections once a particular threshold is exceeded.
Organizations that relay email through known servers or communicate frequently with known partners can and should add the IP addresses of those trusted relays and good email servers to the Rate Control exemption list.
There’s several things we can do to control the rate at which Exim accepts connections. At Main >> Service Configuration >> Exim Configuration Editor you want to set:
Dictionary attack protection – On
Reject remote mail sent to the server’s hostname – On
Ratelimit suspicious SMTP servers – On
SpamAssassin™: ratelimit spam score threshold – <choose a score, I use 10. # This limits the rate a an IP can send mail at to 1/hr if the sent mail scores above the value set.
Log sender rates in the exim mainlog – On
Maximum message recipients (soft limit) # This isn’t set by default
Maximum message recipients before disconnect (hard limit) # This isn’t set by default
I also suggest putting in an SMTP banner delay to help stop ratware. At At Main >> Service Configuration >> Exim Configuration Manager, change “Introduce a delay into the SMTP transaction for unknown hosts and messages detected as spam.” to On.
Step 3: IP Reputation Analysis
- Barracuda Reputation. Barracuda Reputation is maintained by Barracuda Central and includes a list of IP addresses of known, good senders as well as known spammers. Updates to the Barracuda IP Reputation database are delivered to the Barracuda Spam Firewall via Barracuda Energize Updates.
- External block lists. The Barracuda Spam Firewall enables administrators to take advantage of external block lists which are also known as real-time block lists (RBLs) or DNS block lists (DNSBLs). Several organizations maintain external block lists, such as spamhaus.org.
- Customer-defined policy for allowed IP addresses. The Barracuda Spam Firewall enables administrators to define a list of trusted email servers by IP address. By adding IP addresses to this list, administrators can avoid spam scanning of good email, both reducing processing requirements and eliminating the chances of false positives.
- Customer-defined policy for blocked IP addresses. The Barracuda Spam Firewall also enables administrators to define a list of bad email senders. In some cases, administrators may choose to utilize the IP block lists to restrict specific email servers as a matter of policy rather than as a matter of spam protection.
I’m not sure what the BSF does for HELO checks, but I’m including HELO checks and DNS RBL (Real-time Block List) checks into one step.
There are several settings that now need to be enabled at
Main >> Service Configuration >> Exim Configuration Manager:
Reject remote mail sent to the server’s hostname – On
Require HELO before MAIL – On
Require remote (hostname/IP) HELO – On
Require remote (domain) HELO – On
Require RFC-compliant HELO – On # Requires a HELO to be dotted like server.domain.com instead of just ‘server’
I don’t recommend enabling the options to reject SPF or DKIM failures at SMTP time. I feel they would lead to a lot of false positives. It’s better to ‘score’ these failures later with SpamAssassin instead.
Through trial and error, I’ve found the best RBLs to use are b.barracudacentral.org (BSF’s publicly available list), cbl.abuseat.org, and zen.spamhaus.org. All lists are publicly usable, however with SpamHaus if you run a commercial business you will need to pay. I recommend that if you have over 1000 user accounts, you should really go ahead and use rsync to copy down the lists so that way your MX server(s) are contacting your local RBL server instead of sending all those requests out to other servers on the internet. Setting up an RBL server is beyond the scope of this article, but you can use a program called rbldnsd to do it and can use instructions at the RBL providers to get started.
To enable RBLs in cPanel, to go to Service Configuration »Exim Configuration Manager, then set:
RBL: zen.spamhaus.org – On
(I do not recommend enabling SpamCop as it falses a lot)
Next click on “Manage Custom RBLs”. We now want to add a few custom RBLS. WHM asks for the RBL name, DNS list, and Info URL. On my server I use the following:
Your mileage may vary, but those have worked well for me, especially having b.barracuda and zen.spamhaus. I wouldn’t recommend enabling SpamCop as I’ve seen a lot of false positives on it in the past. The other lists generally will blacklist an IP if it’s a known spam sender/bot, whereas SpamCop will block an legitimate business IP based on one user reporting spam from that sender (which may not actually be spam). In other words, there’s a much higher chance from spamhaus and barracuda that the sending IP is actually a spammer or infected machine.
To use the barracuda list you will need to register at http://www.barracudacentral.org/account/register
which will prevent your nameservers from possibly being blocked
*** IMPORTANT NOTE REGARDING DNS LOOKUPS ***
Some lists like spamhaus don’t return a result if the lookup comes from a public resolver, this is illustrated below:
root@vps [/home/trostfam/.spamassassin]# dig +short @18.104.22.168 22.214.171.124.zen.spamhaus.org
root@vps [/home/trostfam/.spamassassin]# dig +short @126.96.36.199 188.8.131.52.zen.spamhaus.org
Make sure your server is set to use either your datacenter’s resolvers, or you can configure named to accept recursive requests from 127.0.0.1, then you can use 127.0.0.1 as your resolver. If you use spamhaus and a public resolver then no mail will be rejected.
Step 3b: Greylisting
cPanel now supports doing greylisting. Greylisting will temporarily defer new SMTP connections from a sending IP. With greylisting, your server will defer the message with ‘due to greylisting’, then the sending server has to re-try the message. If the sending server sends the message again after your server’s configured retry time, then the message will come through. If it re-sends too fast then the message is deferred again. Greylisting can be configured in WHM at Home >> Email >> Greylisting.
When I enabled greylisting, my caught SPAM went from 25/day to about 3/day, so greylisting does really work to cut down to amount of SPAM that going to come in to your server.
While greylisting is enabled serverwide in WHM, it can be configured on a per cPanel account basis, and even a per domain basis in the cPanel account.
Step 4: Recipient Verification
Many spammers attack email infrastructures by harvesting email addresses. The Barracuda Spam Firewall verifies the validity of recipient email addresses through multiple techniques.
- Protocol compliance. Similar to Sender Authentication, a recipient is first validated for being specified properly. An example of an enforcement policy includes, forcing RFC 821 compliance.
- Custom policies. Organizations can define their policies based on allowed recipient email addresses for which spam scanning should be disabled. They can also define their own block lists based on email addresses. Note that allow lists override block lists.
- LDAP recipient verification. Customers of Barracuda Spam Firewall models 300 and higher can choose to reject messages if the recipient email addresses do not appear in the LDAP directory.
- SMTP recipient verification. By default, the Barracuda Spam Firewall rejects messages if the downstream mail server does not accept mail for that recipient.
- Domain Keys. The Barracuda Spam Firewall enables administrators to inspect email messages for DomainKeys (DKIM) and take action when messages fail signature verifcation.
Now that the sender has given a proper HELO, they are not in a blacklist, now we need to see if the address they are sending to exists. So that you understand the importance of recipient verification, let me give you a quick example situation. Let’s say a spammer is blasting out garbage using the address email@example.com. Without recipient verification, what will happen is that your server will accept the message regardless of whether the recipient exists and try to deliver it. If “firstname.lastname@example.org” is sending to address “email@example.com” and the jane address doesn’t exist, your server will then generate a mailerdaemon bounce and send it back to firstname.lastname@example.org. Unfortunately for the REAL email@example.com, he is going to get a lot of these type of non-existent user messages. Exim will hold the connection open from the sender and contact your mail host to see if firstname.lastname@example.org actually exists. If she does then the message will continue to be checked in Exim (we’re still only on step 4 now), but if email@example.com does not exist, Exim will spit back a Recipient Verify Failed message to the sending host therefore the real John@doe.com never gets a ‘fake bounceback message’
cPanel servers can do recipient verification. The first thing to do is to go to Tweak Settings and check the value of “Initial default/catch-all forwarder destination”. It defaults to “System account” instead of “Fail”. Change this to Fail, that way mail to non-existent users will be rejected at SMTP time and recipient verification will work correctly. Do be aware though that cPanel accounts can change that option in their interface at cPanel >> Default Address.
Step 5: Sender Authentication
Declaring an invalid “from” address is a common practice by spammers. The Barracuda Spam
Firewall utilizes a number of techniques to both validate the sender as well as apply policy.
- Protocol compliance. First and foremost, the sender is validated for being specified properly. Examples of enforcement policies include, forcing RFC 821 compliance or requiring fully-qualified domain names.
- DNS lookup. To prevent senders from faking a “from” domain, a DNS lookup is performed on the sender domain to ensure that the domain exists.
- Sender spoof protection. The Barracuda Spam Firewall has the option to prevent “spoofing” of an organization’s own domain by disallowing emails using that domain name to be sent from outside the organization. Note that sender spoof protection should not be enabled if the organization sends messages from outside their internal email infrastructure (e.g., in the case of marketing bulk-mail services).
- Custom policies. Organizations can define their own allowed sender domains or email addresses. They can also define their own block lists based on sender domains or email addresses. Note that allow lists override block lists.
- Sender policy framework (SPF). SPF is a proposed standard with growing momentum, designed to prevent spoofing of email domains. SPF provides a means for organizations to declare their known email servers in their DNS records so that email recipients can validate the identity of the sender domain based on the IP address of the sending email server. The Barracuda Spam Firewall enables email administrators to block or tag messages on failed SPF checks.
Yes I know, the BSF has Sender Authentication before Recipient Verification. I prefer doing a sender verify check AFTER the recipient verify check, because otherwise, you are checking to see if senders exist when they may be sending to non-existent users on your sysem (ie. dictionary attack). By reversing their order, we’re verifying that a recipient exists before we waste a call out to the sender’s MX to see if they exist. cPanel servers have two options here to consider, the first is “Sender Verification Callouts”, the second is “Sender Verification”. I used to recommend that Sender Verification Callouts be enabled so each sender would be validated, however that now will get your server blacklisted very quickly. I do however still recommend that “Sender Verification” be enabled as that option will just verify the existence of the sending domain without verifying the specific sending email address exists in their MX server.
Step 6: Virus Scanning
Virus Scanning takes precedence over all other Mail Scanning techniques and is applied even when mail passes through the Connection Management layers. As such, even email coming from “whitelisted” IP addresses, sender domains, sender email addresses or recipients are still scanned for viruses and blocked if a virus is detected.
This section will give you MIME checking, file extension blocking, virus scanning, and spam filtering. I used Clam Antivirus because it’s free and I’m familiar with it. You can certainly use whichever one you wish. To enable ClamAV in WHM, go to Main >> cPanel >> Manage Plugins, then check the box “Install and keep updated ” next to ClamAV and click save, that will then install ClamAntivirus and configure Exim to use it.
The next thing is back at the Exim Configuration page, you want to enable the options to do attachment and mime scanning:
System Filter File – /etc/cpanel_exim_system_filter
Attachments: Filter messages with dangerous attachments – On
Step 7: Custom Policy (User Specified Rules)
Administrators can choose to define their own policies, perhaps for compliance or governance reasons, which take precedence over spam blocking rules delivered to the system automatically through Barracuda Energize Updates. The Barracuda Spam Firewall enables administrators to set custom content filters based on the subject, message headers, message bodies and attachment file type. In general, administrators do not need to set their own filters for the purposes of blocking spam, as these forms of rules are delivered to Barracuda Spam Firewalls automatically through Barracuda Energize Updates.
In this section I would recommend that you reject messages that score greater than 15. I’ve found 15 is a good value as I’ve not seen any legitimate mail score over 10. I recommend the following:
SpamAssassin™ reject spam score threshold – 15
Automatically whitelist known mobile device providers – On
There are several Access Lists that you can use to put IPs in to bypass certain checks:
Blacklisted SMTP IPs
# IPs from which SMTP connections are dropped unconditionally
Sender verification bypass IPs
# IPs for which to bypass SMTP-time sender verification checks
Only-verify-recipient SMTP hosts/IPs
# Hosts/IPs for which to bypass all SMTP-time checks except recipient verification
Trusted SMTP IPs
#IPs for which to bypass all SMTP-time recipient/sender/spam/relay checks
Backup MX hosts
# Hosts with reverse DNS from which connections are allowed regardless of rate limits.
Step 8: Fingerprint Analysis
A message “fingerprint” is based on commonly used message components (e.g., an image) across many instances of spam. Fingerprint analysis is often as a useful mechanism to block future instances of spam once an early outbreak is identified. Engineers at Barracuda Central work around the clock to identify new spam fingerprints which are then updated on all Barracuda Spam Firewalls through hourly Barracuda Energize Updates.
There’s several different plugins for Spam Assassin that check against a database of messages reported as spam. We’ll be setting up DCC, Pyzor, Razor, and iXhash. All of those plugins will need to be installed from source as they are not available in the CentOS / Red Hat repositories and cPanel doesn’t support third party repositories.
DCC -Distributed Checksum Clearinghouses
tar -zxvf dcc.tar.Z
loadplugin Mail::SpamAssassin::Plugin::DCC” in /etc/mail/spamassassin/v310.pre
Add these 2 lines to /etc/mail/spamassassin/local.cf:
Pyzor – collaborative, networked system to detect and block spam using identifying digests of messages
yum install python-setuptools
(wget it from sourceforge.net)
tar -zxvf pyzor*.tar.gz
python setup.py build
python setup.py install
You can verify pyzor is able to contact it’s server with the command “/usr/bin/pyzor ping” which should return “public.pyzor.org:24441 (200, ‘OK’)”
loadplugin Mail::SpamAssassin::Plugin::Pyzor” in /etc/mail/spamassassin/v310.pre
Add these 2 lines to /etc/mail/spamassassin/local.cf:
Razor – distributed, collaborative, spam detection and filtering network
(wget it from sourceforge.net)
tar -jxvf razor-agents*.bz2
loadplugin Mail::SpamAssassin::Plugin::Razor2” in /etc/mail/spamassassin/v310.pre
Nothing needs to be added to local.cf for this.
iXhash – Similar in function to the above plugins
(wget it from sourceforge.net)
tar -zxvf ihash*.tgz
cp iXhash/iXhash.cf /etc/mail/spamassassin
cp iXhash/iXhash.pm /etc/mail/spamassassin
/usr/local/cpanel/3rdparty/bin/spamassassin -D IXHASH < iXhash.eml # This will test to see if it works, look for IXHASH in the X-Spam-Status line
Step 9: Intent Analysis
All spam messages have an “intent” – to get a user to reply to an email, visit a Web site or call a phone number. Intent analysis involves researching email addresses, Web links and phone numbers embedded in email messages to determine whether they are associated with legitimate entities. Frequently, Intent Analysis is the defense layer that catches phishing attacks. The Barracuda Spam Firewall features multiple forms of Intent Analysis.
- Intent analysis. Markers of intent, such as URLs, are extracted and compared against a database maintained by Barracuda Central, and then delivered to the Barracuda Spam Firewall via hourly Barracuda Energize Updates.
- Real-time intent analysis. For new domain names that may come into use, Real-Time Intent Analysis involves performing DNS lookups against known URL block lists.
- Multilevel intent analysis. Use of free Web sites to redirect to known spammer Web sites is a growing practice used by spammers to hide or obfuscate their identity from mail scanning techniques such as Intent Analysis. Multilevel Intent Analysis involves inspecting the results of Web queries to URLs of well-known free Web sites for redirections to known spammer sites.
Intent analysis is done with the URIDNSBL plugin. URIDNSBL looks up URLs found in the message against several DNS blocklists. cPanel already enables this option in /etc/mail/spamassassin/local.cf.
Step 10: Image Analysis
Today, image spam represents about one third of all traffic on the Internet. While Fingerprint Analysis captures a significant percentage of images after they have been seen, the Barracuda Spam Firewall also uses Image Analysis techniques which protect against new image variants. These techniques include:
- Optical character recognition (OCR). Embedding text in images is a popular spamming practice to avoid text processing in anti-spam engines. OCR enables the Barracuda Spam Firewall to analyze the text rendered inside the images.
- Image processing. To mitigate attempts by spammers to foil OCR through speckling, shading or color manipulation, the Barracuda Spam Firewall also utilizes a number of lightweight image processing technologies to normalize the images prior to the OCR phase. More heavyweight image processing algorithms are utilized at Barracuda Central to quicklygenerate fingerprints that can be used by Barracuda Spam Firewalls to block messages.
- Animated GIF analysis. In addition, the Barracuda Spam Firewall contains specialized algorithms for analyzing animated GIFs for suspect content.
There are several plugins to do this, I used to use fuzzy_ocr, but it’s no longer maintained. Other plugins that looks to be active haven’t been updated in several years either.
Step 11: Bayesian Analysis
Bayesian Analysis is a linguistic algorithm that profiles language used in both spam messages and legitimate email for any particular user or organization. To determine the likelihood that a new email is spam, Bayesian Analysis compares the words and phrases used in the new email against the corpus of previously identified email.
With cPanel 11.36 paths to certain files have changed, so we need to create a symlink:
ln -s /usr/local/cpanel/3rdparty/bin/sa-learn .
Go to http://iandouglas.com/sa-trainer/ and put in your cPanel account username and domain name. This will create a script you run to train the spam filter, updating the bayesian token database. You can put it in /home/user/bin and then add a cron job through cPanel to run the command daily. For your IMAP accounts, if you put SPAM mail that isn’t caught and tagged with ***SPAM*** in a folder called ‘scan-spam’ and mail that is tagged but shouldn’t be in ‘scan-ham’, the script will then use those messages to train the filter so it performs better. If the user has an existing prefs file then you’ll need to add the contents of the script generated file to it, otherwise you can just copy it to /home/user/.spamassassin/user_prefs, but don’t forget to change ownership of that file!
In each user’s user_prefs file, you may want to adjust the scores for BAYES_95 and BAYES_99 so that mail that has such a large percentage chance of being SPAM is scored higher:
score BAYES_95 5
score BAYES_99 7
Step 12: Rule-based Scoring
Beyond absolute blocks that a single filter can apply, the Barracuda Spam Firewall also includes a sophisticated scoring engine that weighs multiple factors where a single filter may result into restrictive policy. By combining multiple rules with known weightings, the Barracuda Spam Firewall can deliver a strong confidence interval for spam messages. The Barracuda Spam Firewall enables administrators to set global spam scores. Certain models of the Barracuda Spam Firewall also support per domain and per user thresholds.
Spam Assassin custom plugins and tests can be enabled by editing /etc/mail/spamassassin/local.cf. You can also set custom scores in the same file. For cPanel users themselves, they can adjust scores by adding the proper line to /home/user/.spamassassin/user_prefs, but they cannot enable a plugin that way or add custom tests.