- Home
- Data
68,167 sitesGlobal8 topics
Web statistics worldwide
How websites worldwide handle HTTPS, email security, robots.txt, AI crawlers, SEO and more in 2026 — measured on 68,167 sites.
AI crawlersWhich AI bots websites block in robots.txt — GPTBot, ClaudeBot, CCBot, Google-Extended and others.12%Blocks an AI crawler11%Blocks GPTBotAd pixelsAdvertising and conversion pixels found on homepages.23%Ad pixels1%Affiliate trackingads.txtPublishers declaring authorized ad sellers with ads.txt.22%ads.txt23%Ad pixelsCookie consentConsent management platforms and cookies on homepages.20%Cookie consent banner46%Sets cookies on first visitllms.txtAdoption of llms.txt next to robots.txt and AI-crawler rules.15%llms.txt12%Blocks an AI crawlerSocial linksWhich social networks popular websites link to.60%Links to social networks50%Open GraphStructured dataSchema.org types, Open Graph and Twitter cards on homepages.47%JSON-LD structured data50%Open GraphMarketing technologyAnalytics, tag managers, pixels, CRM and testing tools — the marketing stack of popular sites.43%Web analytics40%Tag manager
Our crawler: 68,167 sites · Global · updated Nov 13, 2026 · Methodology
Markets
Markets compared
The same checks on popular sites in seven markets.
| Metric | Global | United Kingdom | Netherlands | Italy | Spain | Japan | Latin America |
|---|---|---|---|---|---|---|---|
| Blocks an AI crawler | 12% | 6.5% | 6.7% | 11% | 11% | 4.5% | 8.9% |
| Blocks GPTBot | 11% | 53% | 46% | 9.7% | 10% | 4.4% | 8.4% |
| Blocks ClaudeBot | 11% | 53% | 46% | 9.4% | 9.6% | 4% | 8.2% |
| Blocks CCBot | 11% | 5.8% | 5.4% | 9.6% | 8.5% | 3.7% | 8.3% |
| ads.txt | 22% | 10% | 9.3% | 19% | 14% | 11% | 13% |
| Ad pixels | 23% | 24% | 14% | 26% | 23% | 20% | 31% |
| Cookie consent banner | 20% | 26% | 20% | 37% | 30% | 3.3% | 8.3% |
| llms.txt | 15% | 13% | 11% | 12% | 13% | 5.9% | 18% |