Scrape + crawl API

Any website → clean markdown.

Sila fetches any page, renders JavaScript and WAF-protected sites in a headless browser, and returns markdown plus the signals around it: forms, integrations, JSON-LD, language and category.

500 free credits every month. No card required.

curl
curl https://api.sila.sh/v1/scrape \  -H "Authorization: Bearer sila_9f2c..." \  -d '{"url":"https://example.com"}'
200 OK
{  "statusCode": 200,  "creditsUsed": 1,  "markdown": "# Example Domain...",  "signals": { ... }}

Extracted from the production crawler behind BubblaV

  • 1 credit per static page
  • 5 per headless page
  • Signed webhooks
  • Kept 14 days
API

Scrape one page. Crawl the whole site.

Two endpoints, one credit model. Start with the playground, then call /v1 from your code.

POST /v1/scrape

Returns when the page is ready. Pick formats: markdown, html, signals, siteInfo, screenshot.

curl https://api.sila.sh/v1/scrape \  -H "Authorization: Bearer sila_9f2c..." \  -d '{"url":"https://shop.example.com",       "formats":["markdown","signals"]}'
{  "creditsUsed": 5,  "renderedViaHeadless": true,  "signals": { "category": "product", "forms": 1 }}

POST /v1/crawls

Starts a durable job. Poll it, page through results, or receive signed webhooks as batches land.

curl https://api.sila.sh/v1/crawls \  -H "Authorization: Bearer sila_9f2c..." \  -d '{"url":"https://docs.acme.dev","limit":500,       "webhook":{"url":"https://you.dev/hook"}}'
Sila-Signature: t=1730736000,v1=5257a8...
{  "id": "crawl_71bc09",  "type": "crawl.page_batch",  "data": { "pagesStored": 50, "cursor": 1 }}
Features

Everything the page tells you, not just its text.

Sila returns clean markdown and the structured signals around it, so your pipeline does not need a second parser.

Markdown plus page signals

Every page comes back as clean markdown with structured data folded in, plus title, description, language, category, images, JSON-LD, detected forms and known integrations.

title: Acme Docsencategory: docs2 JSON-LD1 formintercom

WAF and JavaScript rendering

auto mode tries a plain fetch first, then falls back to a headless browser for challenge pages and SPAs.

fetch403headless200

Sitemap and llms.txt aware crawl

Discovery reads sitemaps and llms-full.txt, then falls back to link discovery. Incremental crawls skip what you already have.

  • sitemap.xml482 URLs
  • llms-full.txt38 sections
  • incremental12 changed, 3 removed

Signed webhooks

Get crawl.started, crawl.discovered, crawl.page_batch and crawl.completed pushed to your URL, signed with HMAC-SHA256 and retried with backoff.

Sila-Signature: t=1730736000,v1=5257a869e7ec...
Pricing

Pay in credits. 1 credit per static page, 5 per headless page.

Start free. Upgrade when your crawls grow.

Free$0forever
500 credits / month1 concurrent crawl, 50 pages per crawl
Pro$19/ month
20,000 credits / month3 concurrent crawls, 2,000 pages per crawl
Scale$99/ month
200,000 credits / month10 concurrent crawls, 25,000 pages per crawl
See full pricing
Compare

How Sila compares.

Same job as Firecrawl, Scrape.do and Jina Reader, with page signals and a credit model that tracks real cost.

Markdown output

SilaYes
FirecrawlYes
Scrape.doYes (output=markdown)
Jina ReaderYes

Page signals

SilaTitle, description, language, category, OG image, images, JSON-LD, forms, integrations
FirecrawlPage metadata (title, description, OG)
Scrape.doRaw response only
Jina ReaderTitle and content

Anti-bot / JS rendering

SilaAutomatic: static first, headless browser on WAF / JS challenge
FirecrawlHeadless browser; stealth proxy mode
Scrape.dorender=true headless browser; not on the Hobby plan
Jina ReaderBrowser-rendered fetch

Credit model

Sila1 credit per static page, 5 per headless page
Firecrawl1 credit per page; extra features cost more
Scrape.doCredits depend on proxy type and render mode; only successful requests
Jina ReaderBilled per output token

From public docs, Nov 2026. Sources: docs.firecrawl.dev/billing, scrape.do/documentation/request-costs, jina.ai/reader.

FAQ

Questions, answered.

Still unsure? Email support@sila.sh.

What counts as a credit?

A page fetched statically costs 1 credit. A page that needed the headless browser, or any request with a screenshot, costs 5. Credits reset on the first of each month (UTC).

Do you respect robots.txt?

Crawls respect robots.txt by default; set respectRobots to false to crawl sites you own. Single-page scrapes never check robots.txt.

How long are crawl results kept?

Crawl pages and errors are kept for 14 days after a crawl finishes, then deleted.

What happens when I run out of credits?

Requests return 402 credits_exhausted. A running crawl stops and keeps the pages it already stored. Credits reset on the first of the month, or upgrade any time.

Can Sila crawl pages behind a login?

No. Sila fetches public pages. It does not send cookies or credentials, so pages behind a login are not crawled.

500 free credits every month.

Create a key in under a minute and scrape your first page.

Get API key