Skip to main content

Web Scraper source

Overview​

The Web Scraper source connector connects to the WebScraper.io API and validates access using an API key.

At this time, the connector is in development:

  • It can authenticate (validate credentials).
  • It does not yet support discovery (catalog) or extraction (syncing data).

Prerequisites​

  • A WebScraper.io account
  • A WebScraper.io API key

Authentication​

This connector authenticates using an API key sent in the Authorization header:

  • Header: Authorization: Token <api_key>

During setup, the connector validates credentials by calling:

  • GET https://api.webscraper.io/api/v1/sitemaps

Setup​

  1. In WebScraper.io, generate or copy your API key.
  2. In your workspace, create a new source and select Web Scraper.
  3. Enter the required configuration fields (below).
  4. Save and test the connection.

Configuration reference​

FieldTypeRequiredDescription
api_keystringYesYour WebScraper.io API key. Used to authenticate API requests via Authorization: Token <api_key>.

Streams​

No streams are currently available.

  • Catalog/discovery: returns an empty catalog.
  • Extraction/sync: not yet implemented (the connector will fail if a sync is started).

Limitations​

  • This connector is under active development and does not yet extract data from WebScraper.io.
  • Only API key validation is supported at this time.