Scraping data behind a login

Scraping a webpage behind a login is achieved by using your login cookie for that site. Retrieving this cookie requires using a 3rd party extension called EditThisCookie. Please review their permissions and policy here.

Here's how to scrape data that is behind a login:

• Install the extension that will be used to retrieve the cookie here

• Navigate to the website that you wish to scrape and login to the site if you are not already logged in

• Once logged in, click the EditThisCookie extension icon and a list of cookies will appear in a popup window. In the menu at the top of this popup window look for the arrow icon pointing outwards (it's the third icon from the right). Click this icon to copy the cookies to your clipboard

• In the Simplescraper dashboard (https://simplescraper.io/dashboard), navigate to your recipe, click edit beside the recipe name and then click 'show advanced options'

• Under the cookies section paste the cookies from your clipboard into the text area (ctrl + v or right-click > paste)

• Save the recipe and then run it. Simplescraper should login on your behalf and scrape the data that you need

• Note: some websites don't use cookies for user authentication and so this method may not always work