If you build Retrieval-Augmented Generation (RAG) pipelines, AI agents, or web-scraping workflows in Python or n8n, you know the pain of messy web data.
Passing raw scraped HTML directly into an LLM prompt or vector database wastes context window tokens on <script>, <style>, <iframe>, and navigation noise—and hurts embedding retrieval accuracy. Writing and maintaining custom BeautifulSoup, Turndown, and regex cleanup scripts across every microservice quickly turns into repetitive boilerplate.
Here is how to turn raw HTML into clean, token-efficient Markdown and normalize messy text in one REST call using the Wordflow Text & Markdown API.
Why Clean Markdown Matters for RAG
- Lower token usage: Stripping inline CSS, tracking scripts, and layout tags often reduces payload size by 50–80% before it hits your LLM.
- Better chunking & embeddings: Headings (
#,##), lists, and code blocks give semantic text splitters clean structural boundaries. - Deterministic output: No LLM hallucination risk during preprocessing—just fast, predictable text transformation.
Step 1: Convert Raw HTML to Clean Markdown
First, grab a free API key by subscribing to the BASIC tier (500 free requests/month) on the Wordflow Text & Markdown API on RapidAPI.
Send a POST request to /html-to-markdown with stripScriptsAndStyles enabled:
import requests url = "[https://wordflow-text-markdown-api.p.rapidapi.com/html-to-markdown](https://wordflow-text-markdown-api.p.rapidapi.com/html-to-markdown)" headers = { "Content-Type": "application/json", "X-RapidAPI-Key": "YOUR_RAPIDAPI_KEY", "X-RapidAPI-Host": "wordflow-text-markdown-api.p.rapidapi.com" } raw_html = """ <article> <h1>Q3 Architecture Review</h1> <script>window.analytics.track('pageview');</script> <style>.nav-banner { display: none; }</style> <p>API latency dropped by <strong>42%</strong> after edge caching.</p> <ul> <li>Zero cold-start overhead</li> <li>Deterministic JSON output</li> </ul> </article> """ response = requests.post( url, json={ "html": raw_html, "headingStyle": "atx", "stripScriptsAndStyles": True }, headers=headers ) print(response.json()["markdown"]) Output
# Q3 Architecture Review API latency dropped by **42%** after edge caching. - Zero cold-start overhead - Deterministic JSON output All tracking scripts and inline CSS are gone, leaving clean Markdown ready for chunking or prompt injection.
Step 2: Sanitize Whitespace and Duplicate Lines
When extracting text from PDFs, emails, or legacy web pages, you often end up with erratic line breaks, extra spaces, and repeated lines. Chain the /clean endpoint right after extraction:
const response = await fetch('[https://wordflow-text-markdown-api.p.rapidapi.com/clean](https://wordflow-text-markdown-api.p.rapidapi.com/clean)', { method: 'POST', headers: { 'Content-Type': 'application/json', 'X-RapidAPI-Key': 'YOUR_RAPIDAPI_KEY', 'X-RapidAPI-Host': 'wordflow-text-markdown-api.p.rapidapi.com' }, body: JSON.stringify({ text: "Chunk one: API latency metrics. \n\n\n\nChunk two: Vector index updated. " }) }); const data = await response.json(); console.log(data); Using It in n8n or Make (No-Code Workflows)
If you orchestrate your AI pipelines visually in n8n or Make:
- Add an HTTP Request node after your web scraper or RSS node.
- Set Method to
POSTand URL tohttps://wordflow-text-markdown-api.p.rapidapi.com/html-to-markdown. - Add headers for
X-RapidAPI-Key,X-RapidAPI-Host, andContent-Type: application/json. - Send the JSON payload below in the request body and pass the returned Markdown directly to your OpenAI, Anthropic, or vector store node:
{ "html": "{{ $json.body }}" } Try It Free
- API & Endpoints (
/html-to-markdown,/clean,/analyze,/transform,/diff): Wordflow Text & Markdown API on RapidAPI - Free Browser Text Tools: Wordflow Web App
What preprocessing steps do you run before feeding scraped documents into your RAG pipeline? Drop your setup in the comments!


