Simplescraper

Troubleshooting ​

Sometimes extracting data doesn't go as smoothly as expected. Fortunately most issues can be easily fixed by following the suggestions below. If you've tried these suggestions and still haven't extracted the data that you need then reach out via chat.

Timeout errors ​

This happens when an element that you have selected to extract cannot be found on the page.

Resolution

  • Check the screenshots (click 'view screenshot' on the logs page) to view the webpage and ensure that all looks as expected

  • Ensure the data that you are scraping is not behind a login. The screenshot should make this clear. If a login is required then follow this guide to scrape data behind logins

  • Ensure that the page loaded correctly. If the screenshot is blank or looks incorrect then this should be clear. If so, contact support

  • If you are familiar with CSS selectors, copy one of the selectors that you have chosen (edit your recipe to see all of the selectors for a recipe) and search for that selector on the webpage using the Inspector. Ensure that the selector exists and if it does not then update the selector with the correct value

  • If you are scraping multiple pages and most of your pages scraped successfully with only one or two showing a timeout error then Simplescraper may have navigated beyond the last page or included a page that does not have elements. You can typically ignore the error in this scenario as long as data was returned

Fields are empty or wrong on some rows ​

A field that has a value on the first row but is empty on the others usually points at one item in the list instead of every item. A field the site shows on only some items, such as a sale badge, is empty on the rest and needs no fix.

  • Remove fields you don't need: on the 'Confirm fields' step, click 'Review' under 'Fields with data on only some rows', tick the fields and remove them.
  • Select the value again: remove the field, click 'Add a field from the page' and click the value on the page.
  • Edit the selector: click 'CSS', then 'Edit selector'. A selector ending in the first item's number, such as a[data-testid="listing-item-details-0"], matches only that item. Change it to a[data-testid^="listing-item-details-"] to match every item.
  • Fix a saved recipe: in the dashboard, click the recipe's '⋯' menu, choose 'Edit', click 'Edit CSS selectors', change the selector and click 'Update recipe'.

The site blocks the scrape ​

A cloud run that returns no rows, a captcha or an access-denied page means the site is refusing automated visits.

  • Run the recipe again: each run reaches the site through a different proxy.
  • Check the recipe's health: click the health pill beside the recipe's name. 'Bot Protection' and 'HTTP Response' show whether the site blocked the run. See Recipe Health.
  • Turn on Site Unblocker: in 'Scrape Health', click 'Enable' on the 'Site Unblocker' row. When it reads 'Unblocker active', later runs use proxies that reach the site. On a free plan the row shows 'Contact support' instead.
  • Scrape in the extension: it runs in your own browser, which many sites accept. See What is browser scraping?.

The side panel does not open or cannot read the page ​

  • Open the panel in the tab you want to scrape: each tab has its own panel, so click the Simplescraper icon again after switching tabs. If the icon is missing, pin the extension.
  • Use an ordinary web page: Chrome blocks extensions on its own pages, such as the new tab page and the Chrome Web Store, so the panel shows 'The page could not be read.' there.
  • No list found: click 'Select a list on the page' and click the list. For a page that holds one record, such as a product page, click 'Single fields' instead.
  • The page did not read: click 'Try again'. If it fails again, reload the page and click 'Start scraping'.

The extension and a cloud run return different data ​

The extension scrapes the page as your own Chrome shows it, signed in and in your location. A cloud run opens it fresh on Simplescraper's servers, from the United States, as a new visitor.

Was this guide useful?