PDF to Markdown: what survives and what does not
What a PDF to Markdown converter keeps (headings, lists, tables, page numbers) and loses, how Word and PowerPoint differ, and how to chunk the result for RAG.
Updated 15 November 2026
Guide · URL to Markdown
There are four ways to turn a URL into Markdown. A command-line converter (pandoc, or trafilatura’s CLI) is fastest for one page. A Python library (markdownify or trafilatura) fits when the conversion lives inside your own code. A hosted API (for example Cloudflare’s /markdown endpoint) adds a real browser for JavaScript pages. A batch tool fits when you have hundreds of URLs and need a clear result for every one, including the ones that fail.
Every fact below comes from the project’s own documentation or package page, read on 15 November 2026.
| Feature | pandoc (CLI) | markdownify (Python) | trafilatura (Python and CLI) | Cloudflare /markdown (API) |
URL to Markdown (ours) |
|---|---|---|---|---|---|
| Current version | 3.14, 10 Nov 2026 | 1.2.3, 30 Jun 2026 | 2.3.1, 6 Oct 2026 | hosted service | hosted on Apify |
| Fetches the URL itself | Yes, “using HTTP” | No, you pass it HTML | Yes, fetch_url or -u |
Yes, in a browser | Yes; browser only when needed |
| Picks the main text | No, converts what it gets | No; you choose tags with strip or convert |
Yes; html2txt() gives all text |
not stated on the endpoint page | extractMode: full or article |
| JavaScript pages | No browser | No browser | No browser | Yes, with a section on JS-heavy pages | Yes, renderMode auto or browser |
| Metadata | No | No | Yes, with_metadata=True |
not stated on the endpoint page | Title, description, language, dates, canonical URL |
| Many URLs | Your own loop | Your own loop | Built-in crawling options | One URL or HTML per call | Up to 1,000 per run, status per URL |
| Cost | Free, runs locally | Free, runs locally | Free, runs locally | Browser time (section 5) | Per converted page (section 5) |
The local tools are the right answer more often than not. The hosted options earn their keep on JavaScript-heavy sites and on long lists where you need to know exactly which pages failed and why.
pandoc reads straight from the web. Its manual says: “Instead of an input file, an absolute URI may be given. In this case pandoc will fetch the content using HTTP” (pandoc manual, read 15 November 2026):
pandoc -f html -t markdown https://go.dev/doc/tutorial/getting-started
pandoc -f html -t gfm -o tutorial.md https://go.dev/doc/tutorial/getting-started
The second line writes GitHub-flavoured Markdown to a file. pandoc converts the whole document, so menus and footers come along, and it does not run JavaScript.
trafilatura has a CLI that keeps the main content only. The -u option takes a URL, and --markdown (or --output-format markdown) selects Markdown (trafilatura docs, read 15 November 2026):
pip install trafilatura
trafilatura -u "https://go.dev/doc/tutorial/getting-started" --markdown
markdownify converts an HTML string; fetching the page is up to you. Its default heading style is UNDERLINED, so set ATX if you want # headings (PyPI, read 15 November 2026):
import requests
from markdownify import markdownify as md
html = requests.get("https://go.dev/doc/tutorial/getting-started", timeout=30).text
markdown = md(html, heading_style="ATX", strip=["script", "style"])
print(markdown[:500])
This gives you exact control over each tag and keeps everything on the page, navigation included. Cut the HTML down to the <main> or <article> element first if you only want the content.
trafilatura fetches, picks the main content and returns Markdown with metadata in one call (quickstart, read 15 November 2026):
from trafilatura import fetch_url, extract
downloaded = fetch_url("https://go.dev/doc/tutorial/getting-started")
markdown = extract(downloaded, output_format="markdown", with_metadata=True)
print(markdown[:500])
Its documentation explains that extract() runs “a cascade of extractors (its own rules, then readability and jusText as fallbacks)”, and that fast=True skips the fallbacks, “making extraction roughly twice as fast at the cost of potentially missing content on difficult pages”.
When pages build their text with JavaScript, a plain HTTP fetch returns a nearly empty shell. A hosted browser solves that. Cloudflare’s Browser Run has a /markdown endpoint that “retrieves a webpage’s content and converts it into Markdown format”. You send url (or raw html) with an API token, and the answer is JSON with the Markdown in result (Cloudflare docs, last updated 26 September 2026, read 15 November 2026):
curl -X POST "https://api.cloudflare.com/client/v4/accounts/<accountId>/browser-run/markdown" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <apiToken>" \
-d '{"url": "https://example.com"}'
You need a Cloudflare account and a token with the “Browser Run - Edit” permission. The endpoint takes one URL or one HTML string per call; looping over a list, retries and a per-page failure report are yours to build.
For a search index, a chatbot or a prompt, the goal is the text a reader needs and nothing else:
For retrieval-augmented generation (RAG), split long pages at headings and paragraphs rather than every N characters. A chunk that cuts a code block or a table in half retrieves badly.
At scale, some pages will not convert, and a good pipeline says which and why. The usual causes are a timeout, an HTTP error status, a page that is empty without JavaScript, a CAPTCHA or bot wall, and a robots.txt rule that asks crawlers to stay out. pandoc lets you set any User-Agent header, but pretending to be a browser to get past a block is a choice about the site’s wishes, not a technical fix. We recommend you identify your crawler and respect robots.txt.
URL to Markdown reads plain HTML first and opens a real browser only when a page turns out to be an empty JavaScript shell. A real input:
{
"startUrls": [
{ "url": "https://go.dev/doc/tutorial/getting-started" },
{ "url": "https://docs.python.org/3/tutorial/introduction.html" }
],
"maxUrls": 100,
"renderMode": "auto",
"extractMode": "full",
"includeImages": true,
"includeLinks": true,
"frontMatter": false,
"chunkSize": 2000,
"timeoutSecs": 30,
"maxConcurrency": 5
}
One result per URL, in the order you gave them. This is a real result from a test run on 13 November 2026, with the Markdown cut short (that run had no chunk size, so chunks is null):
{
"url": "https://go.dev/doc/tutorial/getting-started",
"status": "ok",
"title": "Tutorial: Get started with Go - The Go Programming Language",
"markdown": "# Tutorial: Get started with Go\n\nIn this tutorial, you'll get a brief introduction to Go programming. ...",
"wordCount": 1041,
"chunks": null,
"metadata": { "lang": "en", "canonicalUrl": "https://go.dev/doc/tutorial/getting-started" },
"fetchedWith": "http",
"httpStatus": 200,
"crawledAt": "2026-11-13T07:49:36.789Z",
"error": null
}
The status is ok, robots_disallowed, blocked or error. A failed result carries an error code such as TIMEOUT, HTTP_ERROR or EMPTY_CONTENT and a plain message. With chunkSize set (500 to 100,000 characters), chunks is a list of Markdown strings split at headings and paragraphs. fetchedWith shows whether the page needed the browser. Run it on the Apify Store, or schedule it daily to keep a knowledge base current.
Limits: public pages only (no logins, cookies, forms or paywalls); it stops at CAPTCHAs and bot walls and does not hide who it is; up to 1,000 URLs per run and at most 5 million characters of Markdown per result. The crawler identifies itself as TanForgeBot. For PDF, Word and PowerPoint files, use PDF, Word and PowerPoint to Markdown instead.
All read on 15 November 2026. Undated official pages are the current versions on that day.
What a PDF to Markdown converter keeps (headings, lists, tables, page numbers) and loses, how Word and PowerPoint differ, and how to chunk the result for RAG.
Updated 15 November 2026
BuiltWith alternatives compared on public facts: price model, free tier, bulk lookups, change alerts and API, with every figure dated and linked to its source.
Updated 15 November 2026