Website to Markdown for LLM & RAG: Pages or Whole Sites
Convert web pages or whole websites to Markdown for LLM and RAG workflows. Pay per page.
Overview
Converts individual web pages or whole websites (blogs, documentation portals) into clean, RAG- and LLM-ready Markdown by extracting only the main content using Mozilla Readability and a full-page extraction mode that removes navigation, headers, footers and banners; preserves semantic structure by turning HTML tables into GitHub-flavored Markdown tables and normalizing links and images to absolute URLs; captures structured metadata per page (title, description, author, publish/modify dates, language, canonical URL, word count and similar page-level attributes); supports both URL-list processing and site crawling (breadth-first crawl with configurable depth and per-site limits, regex include/exclude patterns and optional sitemap discovery); runs over HTTP without a browser using an in-memory frontier for higher throughput; outputs one record per page containing converted Markdown plus associated metadata optimized for indexing, chunking, embedding and ingestion into LLM pipelines, RAG indexes and vector databases.
Use cases
- Build a knowledge base or RAG index from documentation or a help center.
- Feed competitor blogs or news to an LLM for summarization.
- Create fine-tuning or evaluation datasets from web content.
- Archive articles as Markdown for note-taking apps (e.g., Obsidian, Notion).
Pricing
| Page scraped | $0.003 per event (3 USD per 1,000) |
|---|---|
| Billing | Charged by Apify to your Apify account; platform usage may apply |
How to use
Open on Apify Store
Sign in to Apify or create a free account.
Configure the input
Fill in the input form; each field is documented on the Apify page.
Run and export
Start the run and download results as JSON, CSV or Excel, or use the API.
