home / mcp / claude web scraper mcp server

Claude Web Scraper MCP Server

Bridges Claude with a local eGet web scraper to fetch and summarize web content via a local API.

Installation

Add the following to your MCP client configuration file.

Configuration

{
  "mcpServers": {
    "vishwajeetdabholkar-eget_mcp": {
      "command": "/absolute/path/to/claude-scraper-mcp/.venv/bin/python",
      "args": [
        "/absolute/path/to/claude-scraper-mcp/scrape_mcp_server.py"
      ]
    }
  }
}

This MCP server lets Claude for Desktop connect to a locally running eGet web scraper, enabling you to extract web content and have Claude summarize or query it directly through a simple local API.

How to use

You install and run the local eGet web scraper, start the Claude MCP server, and configure Claude for Desktop to talk to the local Python interpreter running the MCP script. Once set up, you can ask Claude to scrape a website like “Scrape the content from https://example.com and summarize it” or request information about a site it should fetch via the local API.

How to install

Prerequisites you need before starting are: Claude for Desktop, Python 3.7+, and the eGet web scraper you’ll run locally.

# 1. Set up eGet Web Scraper
# Clone the eGet repository
git clone https://github.com/vishwajeetdabholkar/eGet-Crawler-for-ai
cd eGet-Crawler-for-ai

# Set up and run eGet according to its instructions
# (typically using Docker or local Python installation)

# Verify the API is running (default: http://localhost:8000/api/v1/scrape)

### 2. Set up the MCP Server

# Create project directory
mkdir claude-scraper-mcp
cd claude-scraper-mcp

# Set up UV and virtual environment
uv venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate

# Install dependencies
uv add "mcp[cli]" httpx

# Create the MCP server script
touch scrape_mcp_server.py

### 3. Configure Claude for Desktop

# On macOS
mkdir -p ~/Library/Application\ Support/Claude/

{
  "mcpServers": {
    "scrape-service": {
      "command": "/absolute/path/to/claude-scraper-mcp/.venv/bin/python",
      "args": [
        "/absolute/path/to/claude-scraper-mcp/scrape_mcp_server.py"
      ]
    }
  }
}

Replace the paths with the actual absolute paths to your virtual environment and script. After saving, restart Claude for Desktop to apply the new MCP server configuration.

Additional notes

The local MCP server is designed to route Claude’s requests through the eGet web scraper running on your machine. Ensure the eGet API is reachable at the configured URL and that Claude’s Python interpreter points to the same virtual environment where your MCP server script and dependencies reside.

If you need to modify how Claude requests content, update the source script paths accordingly and restart Claude for Desktop after making changes.

Available tools

scrape

Sends a URL to the local eGet API and returns the scraped content, enabling Claude to summarize or analyze the page.