How to use Smart Extract

Tell the AI what data you want — it finds and extracts it for you.

Simplescraper Smart Extract uses AI to accurately extract data and generate reusable CSS selectors from any website. All that's required is a URL and a data schema that lists the properties you wish to extract.

Smart Extract can be accessed via:

• The Simplescraper dashboard - click 'Get new data' in the sidebar, then 'Scrape webpages'. Smart Extract powers the 'Structured data' output there
• Programmatically via the API

Smart Extract + CSS selectors

Smart Extract returns valid CSS selectors, making it easy to convert AI-powered extractions into regular scrape recipes. Start with AI to quickly identify and validate the data you want, then switch to standard scraping for speed, accuracy, and scale.

This combines the flexibility of AI with the reliability of traditional scraping - meaning no hallucinations or context limits.

Dashboard Usage

• Open Get new data in the dashboard sidebar and select the 'Scrape webpages' tab
• Under 'Website to scrape', choose 'Single URL' and enter the URL of the page you wish to extract data from. To run the same extraction across many pages, choose 'Multiple URLs' and paste one URL per line (up to 50 per run)
• Keep 'Data required' set to 'Structured data'
• Under 'Values to extract', enter a data schema (comma-separated list of properties you wish to extract)
• Click 'Get data'
• After a few seconds, your results appear as a table. Switch between the 'Data' and 'CSS selectors' views to see the extracted values and the selector behind each field - when Simplescraper finds the data in the page's network API, the selector view shows an API field path instead
• Click 'Download data' to save the results as CSV
• Click 'Save as scrape recipe' to convert the extraction into a regular scrape recipe, allowing you to scrape at scale using Simplescraper

The 'Scrape webpages' tab also offers two options that work differently from Smart Extract: the 'Markdown' output returns each page as clean markdown text instead of structured rows (see Extracting websites to Markdown), and 'From a recipe' re-runs one of your existing scrape recipes.

API Usage

• API docs can be found here: https://simplescraper.io/docs/api-guide#post-smart-extract.

Tips on writing your data schema

• The schema provided should be a short, accurate list of each of the visible data points on the website that you wish to extract.

  • For example, if extracting data from a jobs board: 'Role, salary, location, job type, company, description, experience required' is a good schema.

• Including a hint of what data is being extracted can increase accuracy.

  • For example, instead of: 'name, location, price, size, bedrooms, bathrooms', including a reference to the type of data can improve results. Example: 'property name, location, price, size (sqm), bedrooms, bathrooms'.

• A schema is not a prompt.

  • This works: "title, old price, current price, discount, review count, description, num capsules, rating".
  • This does not: "visit the website and extract everything on the page beginning with A".

---

Example schemas

Use these as inspiration for your own extractions:

Site	Schema
Car auctions (carsandbids.com)	car name, details, time remaining, bid price, location
Product page (nike.com)	price, old price, name, num of colors
Job board (Microsoft careers)	job title, location, remote possible, description
Real estate (realestate.com.au)	price aud, price us, location, size (m2)
Social profile (x.com)	name, @tag, tagline, joined date, link, number of posts, top tweet text

Workflow example: Using Smart Extract to create a scrape recipe

One powerful way to use Smart Extract is as a starting point for creating a regular scrape recipe. This gives you the flexibility of AI to quickly identify the data you want, and then the speed, accuracy, and reliability of standard scraping for production use.

The idea: extract data and selectors → review → save as recipe → scale.

Step 1: Extract data and selectors using Smart Extract

Use Smart Extract to extract the data you want from a page. You'll get both the structured data and a list of CSS selectors for each field.

async function runSmartExtract(apiKey, url, schema) {
  const response = await fetch('https://api.simplescraper.io/v1/smart-extract', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${apiKey}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({ url, schema })
  });

  const data = await response.json();
  return data;
}

Example usage:

const apiKey = 'YOUR_API_KEY';
const url = 'https://example.com/product-page';
const schema = 'product name, price, description, rating';

const result = await runSmartExtract(apiKey, url, schema);

Step 2: Review the results

You'll receive two important things in the response:

• data: an array of structured values (the extracted data)
• selectors: a list of CSS selectors for each field you requested

If the selectors look good, you're ready to convert them into a scrape recipe.

Step 3: Create a recipe using the returned selectors

Once you're happy with the fields and selectors, use them to create a new recipe. This lets you run scalable scrapes using Simplescraper's standard scraping engine.

async function createRecipe(apiKey, name, url, selectors) {
  const response = await fetch('https://api.simplescraper.io/v1/recipes', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${apiKey}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      name: name,
      url: url,
      selectors: selectors
    })
  });

  const recipe = await response.json();
  console.log('Recipe created:', recipe);
  return recipe;
}

Example usage (continued):

const recipeName = 'Product Page Scraper';
await createRecipe(apiKey, recipeName, url, result.selectors);

So start with Smart Extract to quickly find the right selectors, then save them as a recipe for fast, reliable scraping at scale. If the site changes later, you can run Smart Extract again and update the existing recipe using PUT /recipes/:recipeId - no need to manually update your recipes.

Notes:

Smart Extract is in beta and may not be 100% accurate. If you encounter any issue or incorrect data please contact us via chat