One of our customers has been on the receiving end of a sustained DDoS attack this week. Not from a sophisticated exploit or a criminal botnet, but from aggressive scraping by Tencent’s crawler operating under AS132203.
The attack was generating roughly 20 times the normal number of requests, and it was relentless. In practical terms, that kind of traffic hammers server resources, drives up hosting costs, and can take a perfectly healthy site to its knees.
Part of the problem: an infinite calendar
Part of what made this so bad is an architectural quirk that many sites share without realising it: a web calendar.
Web calendars are a classic trap for badly-written bots. A calendar can generate an essentially infinite number of valid-looking URLs; every day, week, month, and year, going forwards and backwards indefinitely. A sensible crawler respects robots.txt, reads <meta name="robots"> tags, and understands when it’s going in circles. Tencent’s bot did none of these things. It just kept crawling, and crawling, and crawling.
The fix: block by ASN on Cloudflare
Thankfully, Cloudflare gives us a clean solution here. Rather than playing whack-a-mole with individual IP addresses, you can block an entire Autonomous System Number (ASN) in one go. All traffic served by AS132203 – the ASN assigned to Tencent – is now permanently blocked for our customer’s site.
If you’re seeing a similar surge in bot traffic, or would like to pre-emptively prevent one, we’d suggest you do the same.
We’re not anti-scraping
To be clear: we’re not against web scraping in principle. Search engine crawlers, accessibility tools, legitimate research bots – these are all fine, and they generally know how to behave. Respect rate limits. Honour robots.txt. Don’t hammer a server with the same request over and over.
Causing 20 times the normal traffic volume isn’t scraping, it’s abuse. And it’ll be treated as such.
If you’re concerned about your own site’s exposure to this kind of traffic, get in touch – we can help you review your server and Cloudflare configuration, and put the right protections in place.