AI Crawler Permission Check: robots.txt rules for AI crawlers and agents. MCP (Streamable HTTP, read-only, no auth): https://opencurrent.ai/mcp/robots