SiteCheck · api.sitecheck-api.workers.dev

For eleven AI crawlers (GPTBot, ClaudeBot, PerplexityBot…

Google-Extended and others): allowed or blocked for the page's path under robots.txt, plus the noai and noimageai opt-outs in meta and X-Robots-Tag, and whether llms.txt exists. Reads the page as static HTML; no JavaScript runs.

Ready now2 calls · 2 buyers in 30 days

Input

Sample response

Published by the provider. Your run returns a fresh response.

response8 keys

checkedAt"2026-10-14T07:09:42.733Z"

crawlers3 items

0{ allowed, name }2 keys

1{ allowed, name }2 keys

2{ allowed, name }2 keys

llmsTxttrue

optOut4 keys

metaRobotsnull

noaifalse

noimageaifalse

xRobotsTagnull

path"/"

robotsTxt1 key

existstrue

source"https://developers.cloudflare.com/robots.txt and the page (static HTML, no JavaScript run)"

url"https://developers.cloudflare.com/"

responseobject, 8 keys

Your agent can do this on its own

Add the MCP URL in Claude Code, Cursor or Codex. Your agent can run Tools within its spending limits, or ask for approval each time. Connect your agent

Connect your agent

Add one MCP URL and your agent can find and run this tool within its spending limits. Choose “Ask each time” if you want to approve every Call.

MCP URL

https://app.aecon.ai/mcp

Terminalbash

claude mcp add --transport http --scope user agentic-economy "https://app.aecon.ai/mcp"

In Claude Code, open /mcp, select agentic-economy, then choose Authenticate.

Track record

Paid calls, last 30 days: 2
Buyers, last 30 days: 2
Last paid call: 7 days ago

Reported by the x402 directory.

A$0.0029 per call