- Home
- Data
68,769 sitesGlobal8 topics
Web statistics worldwide
How websites worldwide handle HTTPS, email security, robots.txt, AI crawlers, SEO and more in 2026 — measured on 68,769 sites.
AI crawlersWhich AI bots websites block in robots.txt — GPTBot, ClaudeBot, CCBot, Google-Extended and others.10%Blocks an AI crawler9.6%Blocks GPTBotAd pixelsAdvertising and conversion pixels found on homepages.23%Ad pixels1.1%Affiliate trackingads.txtPublishers declaring authorized ad sellers with ads.txt.22%ads.txt23%Ad pixelsCookie consentConsent management platforms and cookies on homepages.19%Cookie consent banner47%Sets cookies on first visitllms.txtAdoption of llms.txt next to robots.txt and AI-crawler rules.15%llms.txt10%Blocks an AI crawlerSocial linksWhich social networks popular websites link to.60%Links to social networks50%Open GraphStructured dataSchema.org types, Open Graph and Twitter cards on homepages.46%JSON-LD structured data50%Open GraphMarketing technologyAnalytics, tag managers, pixels, CRM and testing tools — the marketing stack of popular sites.43%Web analytics40%Tag manager
Our crawler: 68,769 sites · Global · updated Oct 23, 2026 · Methodology
Markets
Markets compared
The same checks on popular sites in seven markets.
| Metric | Global | United Kingdom | Netherlands | Italy | Spain | Japan | Latin America |
|---|---|---|---|---|---|---|---|
| Blocks an AI crawler | 10% | 6% | 6.4% | 11% | 11% | 3.9% | 7% |
| Blocks GPTBot | 9.6% | 47% | 45% | 7.4% | 8.2% | 3.7% | 5.5% |
| Blocks ClaudeBot | 8.6% | 46% | 45% | 6.9% | 7.6% | 3.4% | 5.2% |
| Blocks CCBot | 9.4% | 4.9% | 4.7% | 8.6% | 8% | 3.1% | 5.4% |
| ads.txt | 22% | 11% | 9.4% | 19% | 14% | 11% | 12% |
| Ad pixels | 23% | 26% | 16% | 27% | 24% | 21% | 33% |
| Cookie consent banner | 19% | 27% | 21% | 37% | 30% | 3.2% | 8% |
| llms.txt | 15% | 17% | 14% | 14% | 14% | 7.4% | 18% |