Simplescraper

Simplescraper's MCP Server: AI managed Web Scraping ​

Model Context Protocol (MCP) is a standardized way for AI applications like Claude and ChatGPT to interact with web apps like Simplescraper.

By connecting to Simplescraper's MCP server, you can scrape websites, manage recipes and analyze scraped data from any LLM application, using text and speech.

Let's learn more about what's possible and how to connect.

Simplescraper's remote MCP server URL:

https://mcp.simplescraper.io/mcp


Why Use Simplescraper's MCP Server? ​

Connecting your LLM application to Simplescraper allows you to:

  • Scrape websites through conversation: For example, typing "run my Twitter scrape recipe and show the results" is all it takes to trigger a scrape tasks and display the response
  • Natural language control: No need to remember API endpoints or parameters
  • Instant data analysis: Have AI analyze, summarize, or transform your scraped data
  • Automated workflows: Let AI handle complex multi-step scraping tasks

Quick Start: Adding to Claude ​


Claude ​

  1. Open Claude Desktop settings
  2. Navigate to Settings (bottom-left) → Integrations
  3. Click "Add MCP Server"
  4. Enter the following details:
    • Name: Simplescraper
    • URL: https://mcp.simplescraper.io/mcp
  5. Click through the authentication flow to connect your Simplescraper account
  6. Start using natural language to control your scrapes!

Claude Integrations availability

At the time of writing Integrations, is only available on the Max plan. However it should be more widely avaiable soon.


Other AI Applications ​

Most AI applications that support MCP will have a similar setup process. Look for MCP or "Tools" settings and add our server URL: https://mcp.simplescraper.io/mcp


Available Capabilities ​

The Simplescraper MCP server provides these tools to AI applications:

Recipe Management ​

  • list_recipes - Find recipes by domain, keyword, or recent activity
  • get_recipe - View full details of a specific recipe including selectors
  • create_recipe - Build new scrape recipes with CSS selectors
  • update_recipe - Modify existing recipes (name, URL, or selectors)

Data Extraction ​

  • run_recipe - Execute a scrape to get fresh data
  • smart_extract - Use AI to extract data without CSS selectors
  • get_latest_results - Retrieve the most recent scraped data
  • get_results - Fetch results by specific result ID
  • get_results_history - View past scrape runs for a recipe

Batch Operations ​

  • update_batch_urls - Manage lists of URLs for batch scraping

Example Prompts ​

Here are some ways to interact with Simplescraper through Claude:

Smart Extract a Website ​

"Extract product names, prices, and ratings from https://example.com/products"

Claude will use the smart_extract tool to intelligently identify and extract the requested data.

View Recent Results ​

"Show me the latest results from my Twitter profile scraper"

Claude will find your recipe and display the most recent scraped data.

Update Selectors ​

"The price selector on my Amazon scraper needs updating to .new-price-class"

Claude will update the specific selector in your recipe.

Filter Recipes by Domain ​

"List all my recipes that scrape from linkedin.com"

Claude will show all recipes targeting LinkedIn.

Analyze Scraped Data ​

"Show me my liked tweets and organize them by topic"

Claude will retrieve your Twitter data and categorize it for you.

Manage Batch URLs ​

"Add these 10 product URLs to my product details scraper"

Claude will update the batch URL list for crawler-based scraping.

Advanced Usage Examples ​


Complex Workflows ​

"Check my Amazon price tracker recipe. If prices have changed by more than 10% 
since the last run, create a summary report of the changes."

Multi-Step Operations ​

"Find all recipes that haven't run in the last week, run them, 
and summarize any that return errors."

Data Transformation ​

"Get the latest results from my job listings scraper and format them 
as a markdown table sorted by salary."

Troubleshooting ​


"Claude was unable to connect" ​

  • Ensure you're using the correct server URL: https://mcp.simplescraper.io/mcp
  • Try re-authenticating by removing and re-adding the server

"No recipes found" ​

  • Verify you have recipes created in your Simplescraper account
  • Check that your authentication was successful

Tool errors ​

  • Most errors will include helpful messages about what went wrong
  • Ensure your API key has sufficient permissions for the requested operation

Best Practices ​

  1. Be Specific: The more detailed your request, the better the results
  2. Use Natural Language: No need to memorize tool names or parameters
  3. Iterate: Ask Claude to refine selectors or adjust parameters based on results
  4. Combine Tools: Chain multiple operations for complex workflows

Getting Help ​