Blatant dewlance.com SEO, thrustvps, and HEAD attacks
6th November 2010
Update 4: there are claims that these HEAD-attacks were coming from a malicious dewlance.com customer, and have nothing to do with dewlance itself.
Noticing weird narrow spikes in server load graph, I decided to investigate the most recent one – at 03:50 GMT+2 on Nov. 6, 2010.
The reason was simple: someone issued a few hundred HEAD-requests over a 30 second period to a PHP-based web-application.
All the requests were coming from IP 109.169.59.139, which belongs to the IP range of thrustvps.com:
inetnum: 109.169.58.0 – 109.169.59.255
netname: ThrustVPS_1
descr: Thrust::VPS
country: US
admin-c: RF5058-RIPE
tech-c: RF5058-RIPE
status: ASSIGNED PA
mnt-by: RAPIDSWITCH-MNT
However, it is the referrer string which is more interesting: in all those requests, decorated with varying UserAgents and even operating systems, there was only one referrer – www.dewlance.com.
Initially I thought that was a test of a new DoS attack – really, who would issue dozens of HEAD requests to the same page over a few seconds? However, after seeing that “referrer” string, I now think this is a cheap, blatant, poor and ugly 
I’ll file a complaint with thrustvps if I see that kind of misbehaviour again. All that started on Nov. 4, so there’s still hope people behind this dumb SEO implementation will get fired.
Update 1: they do this every 4 hours since November 4, 2010 (Thursday). This results in loads up to 22, with ~50 apache processes struggling for a few CPU cores:
Update 2: some 20 hours after sending report to abuse at thrustvps.com nothing has changed – still a bunch of HEAD requests every 4 hours. I have written a fail2ban filter+rule to ban anything issuing more than about 1 HEAD request per second. If that rule works as expected – I’ll publish it here.
Update 3: the last HEAD request referring to dewlance occurred at 12:23 GMT on November 8, 2010. I have no idea if that was my complaint, or if that “experiment” just ended naturally.
I have been testing fail2ban rule for false-positives, and it now seems OK. However, I haven’t tested for true positives – I do not know if it will actually block extra HEAD requests (it should).
Put the fragment below into your /etc/fail2ban/jail.local (edit logpath to match your apache logs):
[head-dos]
enabled = true
port = http,https
filter = head-dos
logpath = /var/log/apache2/other_vhosts_access.log
maxretry = 8
findtime = 6
#ban for 25 hours
bantime = 90000
action = %(action_mwl)s
I recommend leaving action as specified for a few weeks to see if you aren’t blocking legitimate requests.
Also paste the fragment below into /etc/fail2ban/filter.d/head-dos.conf:
# Fail2Ban configuration file
#
# Author: bogdan.org.ua
#[Definition]
# Option: failregex
# Note: this regex matches *any* HEAD requests; together with a maxretry=8 and findtime=6 (for example)
# this rule should ban anything issuing more than ~1 HEAD request per second.
#
# sample matching entry:
# bogdan.org.ua:80 109.169.59.139 – - [07/Nov/2010:04:38:33 +0200] “HEAD /2009/10/27/search-and-replace-in-a-mysql-table.html HTTP/1.0″ 200 – “http://www.dewlance.com” “Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.2 (KHTML, like Gecko) Chrome/4.0.221.7 Safari/532.2″
#failregex = ^[^ ]+
-.*”HEAD /.*$ # Option: ignoreregex
# Notes.: regex to ignore. If this regex matches, the line is ignored.
# Values: TEXT
#
ignoreregex =
Update: this rule does work. There were a few false-positives over 2 weeks of testing, so you may need to tune number of requests and time period. After the initial HEAD attacks I’ve seen there were more of these, with other referrer strings – but always a website URL.
Please comment to report improvements/enhancements and problems you’ve encountered with this rule.
November 7th, 2010 at 16:43
I have been investigating similar load average spikes which have become regular this week, several times a day. I implemented a simple monitoring script a week back that sends me an email if the load average spikes above a defined threshhold. Yesterday I got the provider to install a snapshot utility. It took the combination of both to identify the same offender as you found.
Undoubtedly I can add the IP to a no access list. I’m just curious what action you took, aside from filing a complaint?
November 7th, 2010 at 21:44
I have just sent a complaint to ThrustVPS abuse address, including 1.7MB log file of all these requests since November 4.
While waiting for their response I’ll write a fail2ban rule with a day-long bantime against this type of requests. I think limiting to 10 HEAD requests every 10 seconds from a single IP is OK.
When/if those fail2ban rules are ready, I may as well post them here.
November 9th, 2010 at 19:20
Bob – I’ve posted fail2ban rules, if you still see those spikes.
November 10th, 2010 at 7:45
I can’t monitor my client accounts because of Windows not provide any monitor software or tool.
But Client account has been removed.
November 10th, 2010 at 7:55
Sorry for Inconvenience, Also I get attacks from some IPs but all area blocked by my Firewall(CSF)
Please Install this software on your VPS:
1. CSF
2. Mod Security
and use suPHP
Install Instruction: http://configserver.com/free/csf/install.txt
MOD Security: http://www.modsecurity.org/
CSF inform me if someone try to send lots of comments on my blog, try to login, etc.
We’re not doing any illegal works for getting visitors.
I spend money on Advertisements then why I want to block my Site in search engine for spamming.
If you need any help then please contact me
November 11th, 2010 at 1:18
Dear Kunnu Singh, I know it’s hard to monitor for outgoing attacks, and I am inclined to believe that it was just a malicious client.
There is one thing I do not understand, though: is that referrer string added automatically to all outgoing HTTP requests? How is that possible?
November 11th, 2010 at 7:49
I don’t know why its use my domain name in referrer but this client is trying hack my server.
I get a lots of failed login alert on my WHM and Gmail(password reset email)
I have a some old log entries
Log entries:
IP REMOVED – root [07/11/2010:03:26:19 -0000] “POST /login/ HTTP/1.1″ FAILED LOGIN whostmgrd: user password incorrect
IP REMOVED – [07/11/2010:03:26:37 -0000] “POST /login/ HTTP/1.1″ FAILED LOGIN whostmgrd: user name not specified or invalid user
IP REMOVED – root [07/11/2010:03:27:09 -0000] “POST /login/ HTTP/1.1″ FAILED LOGIN whostmgrd: user password incorrect
IP REMOVED – root [07/11/2010:03:27:25 -0000] “POST /login/ HTTP/1.1″ FAILED LOGIN whostmgrd: user password incorrect
IP REMOVED – root [07/11/2010:03:27:34 -0000] “POST /login/ HTTP/1.1″ FAILED LOGIN whostmgrd: user password incorrect
This person is Trying to reset my Gmail password:
I get this email on my personal email account: “Google Password Assistance”
November 11th, 2010 at 12:59
Ok, I’ve read your personal email to me and will update the post text to reflect your position.