Simplescraper

Bulk scraping in the browser ​

To scrape many pages that share a layout, such as every product page in a catalogue, run one saved recipe on a list of URLs. The extension visits each address in turn in your browser, so pages behind your own login work too, up to 5,000 URLs per run.

For scheduled or background bulk scraping, see Bulk scraping in the cloud.

Before you start ​

  • A saved recipe that works on one of the pages you want to scrape. See Scraping and saving recipes.
  • A list of URLs: your own, or the links another recipe collected.

Run a recipe on a list of URLs ​

  1. In the extension, open 'My recipes' and click the recipe.
  2. Choose 'A list of URLs'.
  3. Add the URLs:
    • Paste: paste them into the box, one per line. The counter shows how many URLs it found and skips anything that is not a web address.
    • CSV: click 'CSV' and choose a file. Every web address in the file is added.
    • Import from results: add the links another recipe collected, as described below.
  4. Click 'Run now'. The panel switches to the 'Scrape' tab and visits each address in turn in the current tab.

For a recipe that uses 'Load more' or 'Scroll', tick 'Navigate N pages on each URL' to page through each address the way the recipe does on its start page. Unticked, it scrapes only the page at each URL.

Import URLs from a previous scrape ​

This chains two recipes: the first scrapes a list of links, such as job postings, and the second scrapes the details from each linked page.

  1. Run the recipe that scrapes the list of links.
  2. Open the detail recipe from 'My recipes', choose 'A list of URLs' and click 'Import from results'.
  3. Under 'Run', pick the run that collected the links. If it has more than one column of links, choose one under 'Links column'.
  4. Click 'Add N URLs', then 'Run now'.

The list shows runs made in this browser.

Credits ​

The pages of the whole list count as one run. Up to five pages use no credits and need no account. A longer list uses 1 credit per page, the first five included; the line under 'Run now' shows the most it can cost. See Credits.

Download results ​

When the run finishes, click 'CSV' or 'JSON' to download the rows, or copy them. If you stop a run partway, choose 'Resume unfinished addresses', 'Keep rows' or 'Start again'.

Earlier runs stay in the recipe's '⋯' menu under 'Run history', where 'Download these N rows as CSV' saves a run with a 'Page URL' column added.

For questions or help, reach out via chat.