I just upgraded my Web server’s operating system and applications. As part of this, I upgraded to the latest mod_security rule set. I started with mod_security to block referer spam, but I’m also using it to block connections from blacklisted addresses. mod_security has a steep learning curve, and I’ve had to tweak my rules more than once.
With these upgrades, my anti-referer-spam rules stopped working. Incoming requests with referer spam headers were logging to my access log and showing up in my reports. I asked on the mod-security-users mailing list, and was told that mod_security wasn’t supposed to work the way I was using it.
I’m not sure why it worked. I don’t want to roll back my other upgrades. It’s time to reassess the problem.
I can block referer spam with mod_rewrite, but that requires maintaining rewrite rules across a variety of other sites on my server, and integrating referer spam rules with the rewrite rules those servers need. That would quickly become ugly.
I want the mod_security RBL functionality, and I intend to use some of its other functions later, so mod_security will stay on my server. Mod_security is big and complex, mainly because the real world is big and complex, and trying to debug my simple referer spam problem lead me towards some very complicated debugging. So I stepped back and thought, and decided to try an alternative approach.
Apache has the ability to separate logs based on environment variables. Mod_security can set environment variables.
I changed my mod_security rules like so:
SecRule REQUEST_HEADERS:REFERER "(?i:(porn))" deny,status:412,setenv:spam
If a request contains a refering site that contains the case-insensitive string porn, the client will receive a 412 response. (I considered a 406 response, but my research says that 412 is most appropriate. mod_security will also set the environment variable spam to 1.
On the Apache side, I changed my access log configuration:
CustomLog "|/usr/local/sbin/rotatelogs /var/log/mwl/mwl_spam_log.%Y-%m-%d-%H_%M_%S 86400 -300" combined env=spam
CustomLog "|/usr/local/sbin/rotatelogs /var/log/mwl/mwl_access_log.%Y-%m-%d-%H_%M_%S 86400 -300" combined env=!spam
Apache checks the environment for every request before logging. If a log message has the environment variable spam, it is logged to the spam log. If that variable is not set, the request is logged to the regular access log.
I think that in the long run, this setup will be easier to troubleshoot.