Who Is Crawling Us: 2,048 Hits in a Week, With AI Outreading Google Nearly 12 to 1

The One-Sentence Version

Between 9 and 16 September 2026, clawsportbot.io was crawled 2,048 times by 10 distinct known crawlers. AI assistant crawlers accounted for 1,149 of those hits; Googlebot for 98 — AI reads this site 11.7 times more often than Google does — and the single largest AI crawler was neither OpenAI's nor Anthropic's, but ByteDance's Bytespider, at 407 hits.

This is the first instalment of a monthly series. Every number comes from our own server-side record: we report crawls that have already happened, and nothing here is extrapolated to "the web".

The Full List: 10 Crawlers, 2,048 Hits

Split by class: 1,149 hits from AI crawlers, 899 from search crawlers (bingbot 801 plus Googlebot 98). OpenAI's three user agents together account for 199 hits — still less than half of Bytespider on its own.

What Each Crawler's Behaviour Shows

Bytespider: 407 hits, concentrated in three days. It crawled 127, 101 and 176 times on 12, 13 and 15 September — 404 of its 407 hits in three days — made one request on the 10th and two on the 14th, and did not appear at all on the 9th or the 11th. Its targets are just as concentrated: the Chinese homepage /zh-CN 129 times, /zh-CN/updates 46, /zh-CN/for-users 30, /zh-CN/agent-network-protocol 26, /zh-CN/store 19 — against 14 hits on the English homepage. For any site with a Chinese-market audience, the implication is direct: visibility inside the Chinese-language AI ecosystem is not something Google delivers.

ChatGPT-User: 72 hits, and it is not indexing. This user agent fires when a person asks ChatGPT something and the model fetches a page on the spot. Of its 72 hits, 62 landed on the English homepage; the rest are scattered across /predictions (2), /zh-CN/guides/ai-football-prediction-tools (2), and /agent-network-protocol, /for-users and /es (1 each). It appeared on all seven days — 1, 14, 9, 25, 14, 5 and 4 hits — the rhythm of live questions rather than of a crawl job, and the only direct evidence in this dataset of a human being on the other end.

bingbot: 801 hits, tracking our new content. It made 27 requests on the 9th, jumped to 149 on the 10th, then 131, 88, 160 and 182, falling back to 64 on the last day. It also does not pile onto the homepage: its most-crawled single page, /zh-CN/predictions/leeds-united-vs-newcastle-united-1557402, took 17 of its 801 hits, and the rest spread across traditional-Chinese guides and individual match pages — a crawler working through coverage, not watching a front door.

Googlebot: 98 hits, 73 of them on one day. It made 73 requests on 14 September and 2, 6, 11, 1, 2 and 3 on the other six days. Its most-visited pages were /zh-CN/predictions/athletic-bilbao-vs-elche-1570373 (8) and /zh-CN/guides/asian-handicap-explained (7). Ninety-eight hits put Google sixth on a list of ten.

ClaudeBot: 228 hits, the most evenly spread. It appeared on six of the seven days (1, 48, 41, 50, 27, 61), missing only the last. Its top pages took 2 hits each, mixing simplified- and traditional-Chinese match pages — breadth-first coverage, with no page preference worth the name.

PerplexityBot: 206 hits, absent on two days. 65, 34, 37, 22 and 48 hits on the 9th, 10th, 12th, 14th and 15th; zero on the 11th and 13th. No single page took more than 4 hits.

YouBot: 108 hits across three days, mostly in Spanish. 26, 61 and 21 hits on 12, 13 and 14 September, with /es/contact (15), /es/store (15), /es/agent-network-protocol (14) and /es/predictions (14) taking 58 hits between them.

The remaining three. GPTBot made 40 requests, 39 on 15 September and one on the 9th. OAI-SearchBot made 87 across five days, favouring the English homepage (5) and English match pages. DuckAssistBot visited once all week, on 12 September.

The Week, Day by Day

Seven days, 2,048 hits: the opening day's 97 was the quietest, and no day after it fell below 192.

Which Pages They Go For

The most-crawled page on the site is not the English homepage — it is the Chinese one, 140 hits to 97. And 129 of those 140 came from Bytespider alone, while 62 of the English homepage's 97 came from ChatGPT-User. Two front doors on the same site, watched by two entirely different sets of readers.

Underneath that sits a second split: Bytespider's most-crawled pages are all section pages — homepage, updates, store, protocol — while ClaudeBot, OAI-SearchBot, PerplexityBot and GPTBot return to individual match pages. One crawler is reading the shape of the site; the others are reading its entries.

The 25 most-crawled pages account for 641 hits between them, 400 of which sit on paths beginning with /zh-CN. Guides make the list too: how to choose an AI football prediction tool took 16 hits and /zh-CN/guides/choosing-football-data-analysis-site 17. Our public prediction record at /predictions took 23, and its public git mirror stays open for anyone checking the entries.

How We Measured This

The method is short enough to state in full, and it is what makes these numbers worth citing:

• Where it runs: Next.js middleware, which every page request passes through.
• How a crawler is identified: the request's User-Agent is matched, case-insensitively, as a substring against a list of 20 known AI and search crawlers. A match returns a canonical bot name and a class, llm or search.
• How it is recorded: a matched request fires a fire-and-forget GA4 Measurement Protocol event, ai_crawler_hit, carrying three parameters — bot_name, crawler_class and page_path. The page response never waits on it.
• Where it lands: a separate GA4 property that holds crawler traffic and nothing else, so human analytics stay clean.
• Sampling: none. Every matched request is one row, and every figure above is a direct count.

What This Data Cannot Tell You

Stating the limits honestly matters more than the numbers do:

• User-Agent strings can be spoofed. We do not verify by reverse DNS, so "Googlebot, 98 hits" strictly means 98 requests that claimed to be Googlebot.
• A crawler that never visits is invisible to us. Anything outside the 20-name list, or any crawler that simply ignores this site, does not exist in this dataset.
• One week is a short window. Bytespider was absent on two days and 39 of GPTBot's 40 hits fell on a single day; seven days cannot distinguish a fixed rhythm from a coincidence.
• This is one site, not the web. clawsportbot.io is a mid-sized football data analysis site working mainly in Chinese and English, with a Telegram bot attached. Another sector, language or size of site could produce entirely different figures. Treat this as one sample, not a census.

What We Will Track Next Month

Same method, same counting rules, three questions: is Bytespider's three-day burst a monthly rhythm or a one-week accident? Will ChatGPT-User's 72 hits grow — the only figure here that maps to somebody asking a question? And will Googlebot's 98 rise as new content lands?

Keeping the method fixed is what makes month-to-month comparison possible. See you in the next instalment.