PBL Update and Comparisons - April 2009

We'd like to show you what some typical broadband space looks like in terms of spam-sending bots and Policy Block List (PBL) listings. Let's sample a few chunks of IPv4 space, count the spam bots, and map them graphically to visualize what those ranges look like. These are just examples, conveniently chosen based on our experience in dealing with countless ISP ranges and PBL listings. There is wide variability among such ranges with many gradations between ISPs and even between subnets within a single ISP. Remember, these are just examples!

As a starting point, there are 4,294,967,296 addresses in IPv4 (232), roughly two billion of which are publicly routed on what we see as "The Internet". Here are the number of IP addresses in PBL Zone listings as of April 2009, rounded to 10,000:

|  |  |
| --- | --- |
|   ISP PBL-listed IPs: | 138,730,000 |
|   Spamhaus PBL-listed IPs:
 363,170,000 | |
|   Total IPs listed in PBL:
 501,900,000 | |
|   Single IPs removed from PBL zone:
 250,000 | |

Update, 21 May 2009: Active single IP removals are less than 160,000 after a counting error was corrected. Zone data was not affected.

The PBL's listing policy includes all dynamic IP address ranges and any static IPs belonging to an ISP which, by the ISP's own policy, should not be sending mail directly to the Internet. In terms of actual spam sent, PBL ranges vary from none to pure spam. Spamhaus looks at whois, rDNS, SMTP and CBL bot detection rates in making its decisions on ranges to list in PBL.

The following images are density maps of CBL listings per /24 range, with each /16 range displayed on one row (horizontal). The blue outlined grid represents /24 blocks. The light yellow color indicates less than eight bots detected per /24, and bolder colors represent increasing bot counts. While the color scale ends at 80, some /24 blocks have more than 250 botted IPs detected during a week. Some of those could be the same physical computer connecting to multiple IPs over the course of several days. This is the color scale:

Note: Partial map images are shown in this blog due to format restrictions. Click the images to view the full-size map of each IP address range.

Vodacom's 41.0.0.0/11 range (about two million IPs) in South Africa is typical of much dynamic/broadband end-user IP address space. All of it except the first two /16s are listed in PBL by Spamhaus because the ISP clearly marked its static ranges with custom rDNS, and logically did not mix static and dynamic IP address space. It would be even better if they assign generic rDNS to their dynamic ranges with the format *.dynamic.vodacom.co.za (i.e., "dynamic" right-anchored for regex filters). There are no bots in its static ranges. Where there are bots on dynamic IPs, in nearly all cases it is a moderately low count, one or two per /24, and the infected computers seem to be fixed fairly quickly. End-user removals are quite low (under 100 in the /11) possibly due to lower user density, appropriate user expectations and education, or access to reliable servers provided by the ISP for its customers' outbound mail.

Among the most bot-infested IP address ranges on the 'net, Turk Telecom's 78.160.0.0/11 range mixes static and dynamic IP addresses and does not clearly mark such ranges in rDNS, making it difficult to list dynamic IPs accurately. The ISP does not--and possibly cannot due to Turkish telecom law--take effective action to stop bots on users' infected computers, even in static ranges. Bots on TTNet tend spew for weeks or even months without intervention. TTNet dynamic users may not have access to reliable ISP outbound smarthosts, they often try to send mail from servers on dynamic IPs, and as a result there are more PBL removals in that range. There are about 200 active end-user removals in this /11, out of about 900 total. Such removals are quickly patched by various PBL mechanisms, but the churn of removals does allow a small amount of spam to leak through to PBL users. PBL listings in this range are a mix of Spamhaus entries and Turk Telekom's own PBL entries.

January 2010: Since April 2009, when that graphic was compiled, TurkTelecom has worked hard to clean up its network and now has a bot rate very similar to 41.0.0.0/11, above. Congratulations and thank you, TTNET!

Some ISPs in India, Southeast Asia and South America have similarly bad infection rates but for smaller IP address ranges. This 200.64.0.0/11 range is typical of highly fragmented LACNIC IP address space. Static servers are often mixed with dynamics in a single /24, rDNS is often non-existant, stale or unclear, outbound servers may be unavailable or unreliable and many customers are ignorant of basic security practices (firewall, anti-virus, and frequent updates). There are over 1,700 single IPs actively excluded from PBL Zone by end users in this range, and many such holes get patched every day as soon as they send spam.

At the clean end of the spectrum, France Telecom's 90.0.0.0/11 (under Spamhaus PBL policy) and Comcast's 96.64.0.0/11 (listed under Comcast's own policy) have almost no botnet spam emissions. They both have clear and consistent rDNS naming patterns for dynamic ranges, provide education, security resources, and good outbound mail servers for customers, and take effective action to mitigate spam emissions from dynamic ranges. Both of those ISPs also happen to be active in MAAWG and probably follow MAAWG's best practices document on dynamic ranges. Here are examples of their ranges represented graphically:

France Telecom:

Comcast:

By the way, this bot data is available to ISPs. They can use their PBL account to obtain lists of specific, recent bot IP addresses in their Master IP Address Ranges. PBL and XBL data is updated four times per hour. PBL accounts are free but only available to the authoritative ISP for a given IP address range. See the PBL FAQ for more information. See this FAQ for an explanation of CIDR notation (/32, /24, /16, etc).

(Blog article data c. 2009-04-09.)