Skip to content

Minifetch Technical SEO API · minifetch.com

Cheap robots.txt preflight check before you call the Minifetch…

Cheap robots.txt preflight check before you call the Minifetch /extract and /run/seo-page-audit endpoints: returns whether Minifetch may fetch a target URL (allow/deny) plus its crawl-delay. If blocked and the site is not yours try our external proxy routes. If blocked and the site is yours or to set a custom crawl-delay read https://minifetch.com/tutorials/unblock-minifetch Checks the URL as given, does not follow redirects. Supports GET & POST

Ready now50 calls · 16 buyers in 30 days

Input

Optional. If true, bypasses the 24h robots.txt cache and re-fetches robots.txt for this check. Use after editing your robots.txt or crawl-delay (or adding/removing a Minifetch block) to confirm the change took effect.
The target URL to check against robots.txt

Sample response

Published by the provider. Your run returns a fresh response.

response2 keys
results1 item
successtrue
responseobject, 2 keys
Your agent can do this on its own
Add the MCP URL in Claude Code, Cursor or Codex. Your agent can run Tools within its spending limits, or ask for approval each time. Connect your agent

Connect your agent

Add one MCP URL and your agent can find and run this tool within its spending limits. Choose “Ask each time” if you want to approve every Call.

MCP URL

https://app.aecon.ai/mcp
Terminalbash
claude mcp add --transport http --scope user agentic-economy "https://app.aecon.ai/mcp"

In Claude Code, open /mcp, select agentic-economy, then choose Authenticate.

Track record

Paid calls, last 30 days
50
Buyers, last 30 days
16
Last paid call

Reported by the x402 directory.

A$0.0015 per call