Scrape

The Scrape API provides one-click web scraping without manually managing Session lifecycles. The system automatically creates a Session, navigates to the target URL, waits for the page to load, returns the result, and cleans up resources.

Scrape a Web Page

POST /v1/scrape
X-API-Key: <api_key>

{
  "url": "https://example.com",       // required
  "contextId": "ctx_xxx",             // optional, use existing Context
  "proxyId": "prx_xxx",               // optional, use proxy
  "waitFor": 2000,                    // optional, wait ms (default 0)
  "javascript": "return document.title"  // optional, execute JS and return result
}

# Returns
{
  "url": "https://example.com",
  "html": "<!DOCTYPE html>...",        // full page HTML
  "title": "Example Domain",
  "jsResult": "Example Domain",        // execution result of javascript field
  "sessionId": "ses_xxxx"             // Session ID used (auto-stopped)
}

Example: Get Page Title

const result = await client.scrape.scrape({
  url: 'https://news.ycombinator.com',
});
console.log(result.title); // "Hacker News"

Example: Execute JavaScript

const result = await client.scrape.scrape({
  url: 'https://example.com',
  javascript: 'return Array.from(document.querySelectorAll("a")).map(a => a.href)',
});
console.log(result.jsResult); // ["https://...", ...]

Example: Use Context to Maintain Login State

// Assume ctx_xxx already has login state
const result = await client.scrape.scrape({
  url: 'https://your-site.com/dashboard',
  contextId: 'ctx_xxx',
  waitFor: 2000,  // wait for dynamic content to load
});
console.log(result.html); // full page with user data

Example: Use Proxy to Bypass IP Restrictions

const result = await client.scrape.scrape({
  url: 'https://geo-restricted-site.com',
  proxyId: 'prx_xxx',  // US IP proxy
});

Parameters

Parameter	Type	Required	Description
url	string	Yes	Target page URL
contextId	string	No	Associated Context for reusing login state
proxyId	string	No	Route traffic through a proxy
waitFor	number	No	Additional milliseconds to wait after page load
javascript	string	No	JS executed in page context; return value becomes jsResult

Notes

After a Scrape operation, the Session is automatically stopped. If a Context is associated, the browser state is automatically saved to the Context snapshot.