If you’ve tried connecting an AI agent (like Claude Desktop, Cursor, or a custom LangChain loop) to the live web, you’ve probably hit the same two walls:
- Token bloat: Feeding raw HTML into an LLM burns through context windows and ruins reasoning quality.
- 403 Forbidden: Standard
fetchor basic headless browsers immediately get blocked by Cloudflare, DataDome, or dynamic JS rendering.
I built ClearFetch (clearfetch.dev) and an open MCP server (@unblocked-web/mcp) specifically to solve this: turning any protected URL into clean, LLM-ready Markdown in a single tool call.
Here is how to wire it into your MCP client in 60 seconds.
1. Get an API Key
Grab an API key from clearfetch.dev.
2. Add the MCP Server to your config
Add the following to your Claude Desktop (claude_desktop_config.json) or Cursor MCP configuration:
{
"mcpServers": {
"clearfetch": {
"command": "npx",
"args": ["-y", "@unblocked-web/mcp"],
"env": {
"CLEARFETCH_API_KEY": "your_api_key_here"
}
}
}
}
- What happens under the hood?
When your agent needs to read documentation behind a JS wall, check competitor pricing, or research a live page, it calls the ClearFetch MCP tool:
- Anti-bot & TLS fingerprint handling: Bypasses WAFs and renders SPAs automatically.
- Noise stripping: Removes navbars, cookie banners, ads, and tracking scripts.
- Clean Markdown output: Returns structured text ready for RAG or direct context injection.
Example Prompt to Try
"Read the pricing page at [URL] and compare their enterprise tier limits with our current specs."
Instead of failing on a CAPTCHA or choking on 150KB of minified <div> tags, your agent gets clean Markdown and answers immediately.
Check out the docs and playground at clearfetch.dev. Feedback and feature requests for the MCP server are always welcome!