Website Content Extraction API: readable text from any page
Give it up to 20 URLs and get back the readable content of each page as Markdown or text, the title and language, and the contact points found on it. Built for feeding pages into a model, enriching a CRM, or checking what a site actually says.
POST/v1/websites/extract
{
"data": [
{
"url": "https://example.com/",
"final_url": "https://example.com/",
"status": "ok",
"http_status": 200,
"title": "Example Domain",
"language": "en",
"word_count": 28,
"content": "# Example Domain\n\nThis domain is for use in illustrative examples…",
"emails": [],
"contact_form_url": null
}
],
"meta": {
"credits_used": 1,
"source": "websites"
}
}
What you get
Every field in the response.
| Field | Meaning |
|---|---|
status | ok, thin (client-rendered page), parked, http_error, not_html, unreachable, blocked, refused, robots_disallowed |
content, title, language, word_count | Readable content as Markdown or text (output parameter), capped by max_chars |
emails[], named_emails[], socials{}, named_profiles[], whatsapp, phones[], contact_form_url, tech_stack[] | Contact points found on the page, with include_contacts: every address, company mailboxes first, the ones that name a person repeated in named_emails |
links[] | Outgoing links when include_links is true |
final_url, http_status, canonical_url, truncated, fetched_at | What was actually fetched and when |
How it works
What happens between your request and the response.
Validated before fetching
URLs must be http or https with a real host; private and internal addresses are refused and never charged.
Fetched from a datacenter address
No residential proxies for business websites. Redirects are followed by hand with every hop checked, and bodies are capped at 3 MB.
Boilerplate removed
Navigation, footers, scripts and cookie banners are stripped; headings, lists, tables and links are kept in Markdown.
robots.txt is read first
Each site's own rules are checked before its pages are fetched, and a path the site disallows comes back as robots_disallowed with the rule quoted. We sell a product that reads these rules; we follow them too.
One slow site cannot cost you the batch
Every URL has its own deadline. A host that accepts the connection and then says nothing is marked unreachable and the rest of your list answers normally.
Stops on a block
A 403 or a challenge page returns blocked. It is not retried through another address.
Pricing
One credit system for every endpoint.
One credit per page read, up to 20 per call. A URL we could not reach, a site that refused us, an error page, a file that is not a web page, an address our guard refuses and a path the site's robots.txt disallows are marked and free. Pages that need JavaScript come back thin and cost one credit, because they were read.
Run it on Apify.
For bigger runs, or without writing any code, the same engine runs on Apify as the two listings below. Apify bills per result, so there is no subscription and nothing to pay us, and Apify's own free plan includes $5 of usage a month with no card.
Website Email Scraper
Give it a list of website URLs and get the emails, phone numbers, social profiles, WhatsApp number and contact page from each, plus the page as clean text.
Website Content Crawler
Crawl a whole website into clean Markdown, one record per page, with the contacts found along the way. Honours robots.txt.
Questions
Things people ask about the Website Content Extraction API.
Does it render JavaScript?
No. Server-rendered pages, which includes most business websites, WordPress, Shopify and documentation sites, extract fully. A single-page app that renders everything client-side returns status thin with whatever text the HTML carried.
Can I crawl a whole site?
The API takes a list of URLs you already have, and honours each site's robots.txt before fetching. For following links across a whole site, seeded from its sitemap, use the Website Content Crawler listing on Apify, below.
Is the content cached?
No. Every call fetches the page live, so what you get is what the site serves at that moment.