Latest reading, ranked/From the Studio
Top Websites Blocking Common Crawl CCBot and 7 more
What this chart reads·8 series
Show allHide
- Top Websites Blocking Common Crawl CCBotas “CCBot (Common Crawl)”latest 2026-10-01
- Top Websites Blocking OpenAI GPTBotas “GPTBot (OpenAI)”latest 2026-10-01
- Top Websites Blocking ByteDance Bytespideras “Bytespider (ByteDance)”latest 2026-10-01
- Top Websites Blocking Anthropic ClaudeBotas “ClaudeBot (Anthropic)”latest 2026-10-01
- Top Websites Blocking Meta-ExternalAgentas “Meta-ExternalAgent”latest 2026-10-01
- Top Websites Blocking Google-Extendedas “Google-Extended”latest 2026-10-01
- Top Websites Blocking Applebot-Extendedas “Applebot-Extended”latest 2026-10-01
- Top Websites Blocking PerplexityBotas “PerplexityBot”latest 2026-10-01
Sources·8 sources
Show allHide
Top Websites Blocking Common Crawl CCBot. Source: Palanor robots.txt survey. Once a month Palanor reads the public robots.txt file of each of the 10,000 most-linked websites in the Majestic Million (majestic.com, licensed CC BY 3.0), using its own crawler name (PalanorBot). Palanor computes and publishes the shares; the site list and the files are not republished.
Top Websites Blocking OpenAI GPTBot. Source: Palanor robots.txt survey. Once a month Palanor reads the public robots.txt file of each of the 10,000 most-linked websites in the Majestic Million (majestic.com, licensed CC BY 3.0), using its own crawler name (PalanorBot). Palanor computes and publishes the shares; the site list and the files are not republished.
Top Websites Blocking ByteDance Bytespider. Source: Palanor robots.txt survey. Once a month Palanor reads the public robots.txt file of each of the 10,000 most-linked websites in the Majestic Million (majestic.com, licensed CC BY 3.0), using its own crawler name (PalanorBot). Palanor computes and publishes the shares; the site list and the files are not republished.
Top Websites Blocking Anthropic ClaudeBot. Source: Palanor robots.txt survey. Once a month Palanor reads the public robots.txt file of each of the 10,000 most-linked websites in the Majestic Million (majestic.com, licensed CC BY 3.0), using its own crawler name (PalanorBot). Palanor computes and publishes the shares; the site list and the files are not republished.
Top Websites Blocking Meta-ExternalAgent. Source: Palanor robots.txt survey. Once a month Palanor reads the public robots.txt file of each of the 10,000 most-linked websites in the Majestic Million (majestic.com, licensed CC BY 3.0), using its own crawler name (PalanorBot). Palanor computes and publishes the shares; the site list and the files are not republished.
Top Websites Blocking Google-Extended. Source: Palanor robots.txt survey. Once a month Palanor reads the public robots.txt file of each of the 10,000 most-linked websites in the Majestic Million (majestic.com, licensed CC BY 3.0), using its own crawler name (PalanorBot). Palanor computes and publishes the shares; the site list and the files are not republished.
Top Websites Blocking Applebot-Extended. Source: Palanor robots.txt survey. Once a month Palanor reads the public robots.txt file of each of the 10,000 most-linked websites in the Majestic Million (majestic.com, licensed CC BY 3.0), using its own crawler name (PalanorBot). Palanor computes and publishes the shares; the site list and the files are not republished.
Top Websites Blocking PerplexityBot. Source: Palanor robots.txt survey. Once a month Palanor reads the public robots.txt file of each of the 10,000 most-linked websites in the Majestic Million (majestic.com, licensed CC BY 3.0), using its own crawler name (PalanorBot). Palanor computes and publishes the shares; the site list and the files are not republished.
Charts are drawn from Palanor's signal library and recomputed from source data on every view. Transformations (changes, indexing, year-on-year rates) are Palanor's.