Perplexity · search bot · Last updated 2026-05-22
How to allow or block PerplexityBot and Perplexity-User in your robots.txt. Why Perplexity is the highest-volume citation-linker among AI engines and why both user-agents matter.
PerplexityBot is the primary indexing crawler for Perplexity AI. Perplexity also operates Perplexity-User, an on-demand fetcher that retrieves a specific URL when a user mentions or links to it during a query. Both bots respect standard robots.txt rules. PerplexityBot crawls broadly on a paced schedule; Perplexity-User fetches per-query when needed for grounding a real-time response.
Perplexity is the most aggressive citation-linker among AI engines — almost every response includes inline URL citations to the source pages it consulted. Being in Perplexity's index translates directly to traffic when users click those citations. Blocking PerplexityBot removes you from search-grounded retrieval; blocking Perplexity-User stops the on-demand fetches that fuel real-time citation. Most brands should allow both.
BrandCited recommendation
Allow both PerplexityBot and Perplexity-User. For brands prioritising AI-driven traffic over training-corpus inclusion, Perplexity should be at the top of the allowlist — its citation-clicking behavior generates the most measurable referral traffic of any AI engine in BrandCited's scan data.
The exact directive to add to your robots.txt for PerplexityBot. Paste at the end of your file — bot-specific blocks override the wildcard above.
robots.txt
Copy and paste
# Allow both Perplexity crawlers
User-agent: PerplexityBot
Allow: /
Disallow: /admin
Disallow: /api/
Disallow: /dashboard
User-agent: Perplexity-User
Allow: /
Disallow: /admin
Disallow: /api/
Disallow: /dashboardPerplexity is search-grounded — it retrieves live web results for almost every query and cites the source URLs inline. ChatGPT and Claude are primarily knowledge-presence (answering from training) with optional retrieval. For Perplexity, being in the live index matters more than being in the training corpus.
Yes, but conservative defaults are appropriate. Perplexity's crawl pace is moderate — a Crawl-delay above 5 seconds is rarely needed and can slow your index-freshness.
They are independent user-agents. Blocking PerplexityBot removes you from the bulk index but Perplexity-User can still fetch specific URLs on-demand if a user mentions one. To fully opt out, block both.
Yes — User-Agent strings start with "PerplexityBot/" or "Perplexity-User/". Perplexity publishes its IP ranges at https://docs.perplexity.ai/guides/bots for verification against spoofing attempts.
For content-grounded queries Perplexity re-fetches via Perplexity-User in real-time, so changes appear in seconds for URLs users mention. For broad index updates from PerplexityBot, expect days to weeks depending on crawl frequency for your domain.
Cite this guide
BrandCited. (2026). PerplexityBot robots.txt — How to Allow, Block, or Audit. https://www.brandcited.ai/tools/robots-txt-auditor/perplexitybot
Each major AI engine operates one or more user-agents. Configure them in parallel for complete coverage.
robots.txt is one of dozens of AI ranking factors BrandCited audits. Run a free scan to also check schema completeness, llms.txt configuration, content structure, entity recognition, and AI citation share-of-voice.