# 0-9 User-agent: 360Spider Disallow: / # A User-Agent: adscanner Disallow: / User-agent: adsbot Disallow: / User-agent: ahrefsBot Disallow: / User-agent: aiHitBot Disallow: / User-agent: Applebot Disallow: / User-agent: Amazonbot Disallow: / User-agent: anthropic-ai Disallow: / # B User-agent: BacklinkCrawler Disallow: / User-agent: barkrowler Disallow: / User-agent: BLEXBot Disallow: / User-agent: BUbiNG Disallow: / User-agent: Bytespider Disallow: / # C User-agent: CCbot Disallow: / User-agent: ChatGPT-User Disallow: / User-agent: ClaudeBot Disallow: / User-agent: Claude-Web Disallow: / User-agent: Cliqzbot Disallow: / User-agent: coccoc Disallow: / User-agent: cohere-ai Disallow: / User-agent: Cpanel-HTTP-Client Disallow: / User-Agent: crawler4j Disallow: / User-agent: crawler_eb_germany_2.0 Disallow: / # D User-agent: dotbot Disallow: / User-agent: DotBot Disallow: / User-agent: Diffbot Disallow: / User-agent: DataForSeoBot Disallow: / User-agent: DatadogSynthetics Disallow: / User-agent: DomainCrawler Disallow: / # E User-agent: Exabot Disallow: / # F User-Agent: FacebookBot Disallow: / User-agent: facebookexternalhit Disallow: / # G User-agent: GeedoProductSearch Disallow: / User-agent: Google-Extended Disallow: / User-agent: GPTBot Disallow: / User-agent: Go-http-client Disallow: / # H User-agent: HaosouSpider Disallow: / # I User-agent: ICCrawler Disallow: / User-agent: ImagesiftBot Disallow: / User-agent: IndeedBot Disallow: / User-agent: INETDEX-BOT Disallow: / # J # K User-agent: Kraken Disallow: / # L User-agent: Linguee Disallow: / User-agent: LinkStats Disallow: / User-agent: Lipperhey-Kaus-Australis Disallow: / User-agent: ltx71 Disallow: / # M User-agent: magpie-crawler Disallow: / User-agent: meanpathbot Disallow: / User-agent: MegaIndex.ru Disallow: / User-agent: megaindex.com Disallow: / User-agent: Meta-ExternalAgent Disallow: / User-agent: MetaJobBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: mj12bot Disallow: / # N User-agent: netEstate Disallow: / # O User-agent: OAI-SearchBot Disallow: / User-agent: Omgilibot Disallow: / User-agent: Omgili Disallow: / User-agent: oBot Disallow: / # P User-agent: PerplexityBot Disallow: / User-agent: PetalBot Disallow: / User-agent: plukkie Disallow: / User-agent: Python/3.5 aiohttp Disallow: / # Q User-agent: Qwantify Disallow: / # R User-agent: R6_CommentReader Disallow: / User-agent: rogerbot Disallow: / # S User-agent: Screaming Frog SEO Spider Disallow: / User-agent: SearchmetricsBot Disallow: / User-agent: seoscanners.net Disallow: / User-agent: SeekportBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: SEOdiver disallow: / User-agent: SEOkicks-Robot Disallow: / User-agent: SerendeputyBot Disallow: / User-agent: serpstatbot Disallow: / User-agent: SeznamBot Disallow: / User-agent: sg-Orbiter Disallow: / User-agent: Sidetrade Disallow: / User-agent: Siteliner Disallow: / User-agent: sogou spider Disallow: / User-agent: Spiderbot Disallow: / User-agent: SurveyBot Disallow: / # T User-agent: ThumbSniper Disallow: / User-Agent: trendictionbot Disallow: / # U # V-W User-agent: VelenPublicWebCrawler Disallow: / User-agent: WBSearchBot Disallow: / User-agent: wotbox Disallow: / # X -Y-Z User-agent: YouBot Disallow: / User-agent: YoudaoBot Disallow: / User-agent: ZoominfoBot Disallow: /