Crawler access
Public content for discovery. Private charts stay private.
BaziRoute separates traditional search, AI retrieval, user-triggered fetches and model-training crawlers. The policy is explicit rather than hidden in an undocumented robots configuration.
Direct answer
Public educational and tool pages may be crawled and cited. Account, login, saved-chart, report and admin pages remain private. Allowing retrieval does not grant permission to bypass authentication or access another person’s data.
| Crawler | Purpose | Current decision | Boundary |
|---|---|---|---|
| GooglebotGoogle | Search | Allowed | Public pages should remain crawlable for traditional search indexing. |
| BingbotMicrosoft | Search | Allowed | Public pages should remain crawlable for Bing indexing. |
| OAI-SearchBotOpenAI | AI search | Allowed | Allows public content to be discovered by OpenAI search features under current policy. |
| ChatGPT-UserOpenAI | User-triggered access | Public pages only | May fetch a page when a user explicitly asks; private routes remain protected and noindex. |
| GPTBotOpenAI | Model training | Allowed | Current public robots policy does not block training crawlers. The decision is reviewed quarterly. |
| PerplexityBotPerplexity | AI search | Allowed | Public content remains available for retrieval and citation under the current site policy. |
| ClaudeBotAnthropic | Model training | Allowed | Current public robots policy does not block this crawler; provider behavior remains controlled by the provider. |
| CCBotCommon Crawl | Model training | Allowed | Public content is currently crawlable by Common Crawl. This decision can be revised with the copyright policy. |
Private routes
Account, login, saved-chart, AI report and administrator routes are excluded from indexing and require authentication or Access.
Decision record
- Owner:
- BaziRoute engineering and editorial
- Reviewed:
- 2026-10-27
- Next review:
- 2027-01-27
What this policy does not do
- · It does not guarantee indexing, retrieval or citation by any provider.
- · It does not grant access to private reports, accounts or billing data.
- · It does not add hidden prompts or special instructions for AI systems.
- · Provider crawler behavior remains subject to each provider’s implementation.