Stop spammers from exploiting your webserver!

For many years, speaking of "botnet spam" mainly meant speaking about compromised Windows systems. However, in the last few years this assumption is no longer entirely true. Looking at the number of distinct sources, the vast majority of emitters are still about the same as before, but looking at volumes we see that a large part of email spam now comes from abused Linux/Unix systems. Part of this volume is due to password abuse, as we discussed in a previous article, Spam through compromised passwords. Here we talk about the second big mechanism: Abuse of vulnerable webservers.

The current situation is the result of vast numbers of webservers running old, unmaintained and vulnerable web applications; applications that are exploited and used to install abusive software including - but not limited to - spamware. Since these machines are usually servers hosted in datacenters instead of end-user machines sitting behind slow ADSL connections, the amount of spam they can pump out is way higher. That higher bandwidth easily explains why relatively few spam sources can contribute so heavily to the global quantity of spam.

Keeping Linux (and generically *NIX, as the same advice applies to other systems as well) webservers secure is a critical component of the fight against botnet and malware-related spam. All webserver owners should know the 5 critical steps to securing a webserver:

Keep applications updatedDo not install software from untrusted sourcesKnow your applications and secure your filesystemWrap your SMTP daemonBlock direct-to-MX Sending

More on those last two points will be covered in a subsequent post.

The following is an introduction to how to keep webservers as secure as possible and harden them to reduce the chances of spam outbreaks:

Keep applications updated

50 years of computer science have taught us how bugs are an inevitable component of code, and the only way to get rid of known bugs is to update the software and/or apply the proper patches. This should always be the first line of defense, no matter what the applications are. The various Content Management Systems (CMS) commonly used nowadays to create websites are not immune from this rule: WordPress, Joomla, Drupal and any other common platform have had security problems that were used to turn perfectly legitimate websites into massive spam cannons. Those systems provide their own tools to keep the website code up-to-date. Use them! Don't assume that your site will be safe from being compromised just because it is a minor website, unlikely to attract the interest of any hacker. People trying to exploit websites for these purposes are not trying to gain fame or Alexa scores, they just need as many compromised systems as they can get. They are interested in quantity, not quality. They use automated scanning tools to look for any possible site to crack. Every piece of web service (CMS included) shows some sort of signature that can be used to know which piece of software is running, and even search engines can be used to get very comprehensive lists of sites running on a specific platform. It is not too hard to take advantage of that and automatically try to break into a service by exploiting known vulnerabilities for that platform. That kind of activity is the reason for the number of hacking attempts you daily see in your webserver logs.

Do not install software from untrusted sources

Some people compromise their systems with their own hands. Installing web server software (for instance PHP modules such as plugins or themes) that has been downloaded from unofficial locations, or even worse installing pirated software, is likely to bring dangerous malware such as the CryptoPHP backdoor into your system. Criminals routinely use social engineering tricks to induce people to download software that appears to be simply a pirated copy of commercial software, but that in reality contains additional backdoor functionalities that give control of your server to the criminals. Emission of spam is just one of the several nasty effects that could occur as a result.

Know your applications and secure the filesystem

The steps above might not be sufficient to save you if your application was developed internally, is highly modified or is no longer maintained. Knowing your application’s access requirements can help a lot. Try to divide the directories it uses in two groups:

• directories containing code and assets that will not be altered by the application
• directories containing assets that will be altered by the application

Directories in the first group don’t need to be writable by the user the web application is running as. All the web server needs to do is read the directory and/or execute the code, but it does not need to write in there under any circumstance.

Directories of the second group need to be writable but since they do not contain application code there is no reason to allow execution for stuff in there. Depending on whether your application uses PHP, perl, python or any other language, you always have the chance to disable the execution of code in certain locations and directories. Do it, and disable code execution for directories that are writable for the application.

Assume an attacker manages to use your application to upload arbitrary files on your web server. If the location where these files are stored allows code execution, the attacker can add his/her own code on your server and run it. The same applies to directories storing your own application’s code: if these files and directories are writable, the attacker can overwrite your own software to run his/her illicit code instead. In both conditions, you would end up allowing the execution of code provided by the attacker and losing control on what your system is running. Prevent that by strictly controlling access to directories which do not need to be altered by your application.

Wrap your SMTP daemon

If your system runs several websites and you find out it has been used to send spam, the first question you will ask yourself is: “How do I figure out which website got compromised and which script has been sending the spam?”

Several approaches can be used in order to deal with this kind of problem, but the first and perhaps easiest one is wrapping your /usr/sbin/sendmail through a script able to add informative headers about what is injecting the email in your MTA’s queue.

Take your time and search on the Internet for your specific mail server. Even if you have no idea what you are supposed to do you will be able to find at least a few implementations you can use. Use your wrapper to replace the usual sendmail binary and call the original binary from it.

Make sure that local users can not relay through the local SMTP daemon by simply connecting to port 25 on localhost, or all the above would be easy to bypass. Local users should access mail servers using authenticated protocols only. This wouldn't provide knowledge of the page actually sending the email (like the the "sendmail wrapping" approach above can do), but binding each website to a set of credentials will at least allow you to identify who is generating the email.

Block direct-to-MX Sending

In addition to injecting spam in the local MTA queues, abusive web scripts also have another chance: opening a direct connection to the recipients’ MX and pump spam directly. It’s the server-side version of Direct-to-MX botnet spam.

Make sure outgoing traffic for remote port 25/TCP is inhibited to web users. You can obtain that either through kernel security modules (like AppArmor and similar resources) or iptables (“owner” module). The only user allowed to generate direct SMTP connection should be the one on which your local MTA runs.

Variations of the same concept are obviously possible, so choose the solution that better fits your needs and your architecture. A very large portion of today’s spam is transmitted this way, so addressing this point is really important.

So, to summarize the last two points in an ideal configuration:

• Direct-to-MX is blocked entirely.
• The web server is not allowing non-authenticated relay, not even from localhost.
• All email from local users should use a wrapper for the local sendmail binary adding submission details or use port 587 (SMTP AUTH) to connect to a smarthost with userid/password. The smarthost should include the userid in a header.

More information about mail servers will be posted in a subsequent article.

Additional information:

DBL FAQ: Tell me more about the "abused legit" part of DBL?

DBL FAQ: How do I fix the problem? Help for domains listed in "abused legit"

MELANI: Instructions for cleaning up websites (also available in French, German and Italian)

Wordpress FAQ: My site was hacked

Joomla!: Introduction to Joomla! Security

Drupal: Drupal Security Team

Gianmarco Pagani is an engineer and security expert in charge of many Spamhaus systems.