home / mcp / scrapegraph mcp server
Provides enterprise web scraping tools accessed through the MCP interface for AI assistants.
Configuration
View docs{
"mcpServers": {
"scrapegraphai-scrapegraph-mcp": {
"url": "https://scrapegraph-mcp.onrender.com/mcp",
"headers": {
"SGAI_API_KEY": "YOUR_API_KEY"
}
}
}
}You can run ScrapeGraph's MCP Server to enable your language models to perform AI-powered web scraping through a stable, production-tested MCP interface. It offers a suite of tools for converting pages to markdown, extracting structured data, performing multi-page crawls, and orchestrating agentic scraping workflows, all accessible from your MCP client.
Connect to the MCP server using your MCP client of choice, such as Claude Desktop or Cursor. You will access eight enterprise-ready tools that let you convert pages to markdown, extract data with AI prompts, crawl large sites, handle infinite scrolling, render JavaScript-heavy pages, and return results in markdown or structured JSON.
Begin by configuring your client to point at either the remote MCP service or your local MCP server. For remote usage, provide the HTTP endpoint and your API key. For local usage, start the server locally and point your client to the local transport. Then issue natural language prompts like: convert a page to markdown, extract product prices from a page, or crawl a site and summarize findings. The MCP server will execute the requested tool and return the results to your client.
Prerequisites: you need Python 3.13 or higher and a ScrapeGraph API key. You may also use the Smithery deployment flow if you prefer one-command installation.
Step-by-step concrete flow to run locally from scratch:
Clone the repository and install dependencies locally, then run the server. The final starting command is provided below.
Use the hosted MCP server so you do not need to install anything locally. Configure your MCP client with the remote server URL and your API key.
Remote HTTP MCP configuration example (paste this into your client config):
{
"mcpServers": {
"scrape_http": {
"type": "http",
"name": "scrape_http",
"url": "https://scrapegraph-mcp.onrender.com/mcp",
"args": []
}
}
}You can run the MCP server on your machine for development or testing. Ensure you have an API key configured in your environment.
Two common local start methods are shown. Use the final start command that matches your preferred approach.
Always protect your API key. Use environment variables to pass secrets where possible, and keep configuration files secure.
If you encounter issues, verify that your API key is set, ensure the MCP server starts without errors, and check logs for any startup problems. If tools do not appear in your MCP client, confirm the server is running and the client configuration references the correct MCP server name.
This MCP server is designed for production-grade reliability with robust error handling and scalable tooling. You can extend tools or integrate with the Google ADK or other agent frameworks using standard MCP practices.
Transform any webpage into clean, structured markdown format.
AI-powered extraction of structured data from a webpage with optional infinite scrolling.
AI-driven web searches with structured results and optional time filtering.
Fetch page content with optional heavy JavaScript rendering.
Extract sitemap URLs and structure from a website.
Initiate multi-page asynchronous crawling with optional AI extraction or markdown conversion.
Poll for crawl results and retrieve status and data.
Run advanced agentic scraping workflows with customizable steps and schemas.