The Attack of the Spiders from the Clouds
...spiders that would not follow our robots.txt directive not to crawl the site. One of the bots was from Russia, one was from China, and another one was from Korea. There were spiders from places I never heard of, all consuming precious resources and denying our users
So, I did what any Linux admin would do. I used iptables to block the...
