Oxylabs Web Scraping MCP server

Oxylabs MCP server provides a bridge between AI models and the web, enabling them to scrape any URL, render JavaScript-heavy pages, extract content for AI use, bypass anti-scraping measures, and access geo-restricted web data from 195+ countries. This implementation leverages the Model Context Protocol (MCP) to create a standardized way for AI assistants to interact with web content.

Key Features

Web Scraping Capabilities

• Extract data from any URL, including complex single-page applications
• Render dynamic websites using headless browser support
• Bypass sophisticated bot protection systems with high success rate
• Access geo-restricted content from 195+ countries

AI-Ready Data Processing

• Automatically clean and convert HTML to Markdown
• Use specialized parsers for popular sites like Google and Amazon
• Feed data directly into AI models or analytics tools

Available Tools

Web Scraper API Tools

• universal_scraper: General website scraping
• google_search_scraper: Extract results from Google Search
• amazon_search_scraper: Scrape Amazon search result pages
• amazon_product_scraper: Extract data from Amazon product pages

AI Studio Tools

• ai_scraper: AI-powered data extraction from any URL
• ai_crawler: Crawl multiple pages based on a prompt
• ai_browser_agent: Control a browser to achieve specific tasks
• ai_search: Search the web for URLs and their contents

Prerequisites

Before installation, you'll need:

• Oxylabs Web Scraper API Account: Obtain your username and password from Oxylabs (1-week free trial available)
• Oxylabs AI Studio API Key (Optional): For AI-powered tools, obtain your API key from Oxylabs AI Studio

Basic Usage Requirements

• Node.js (v16+) and npx command-line tool (for Smithery CLI)
• or uv package manager (install using this guide)

Local/Dev Setup Requirements

• Python 3.12+
• uv package manager

Installation and Configuration

Using Smithery

1. Go to https://smithery.ai/server/@oxylabs/oxylabs-mcp
2. Login with GitHub
3. Find the Installation section and follow the instructions

# Example for Claude Desktop
npx -y @smithery/cli@latest install @upstash/context7-mcp --client claude --key <smithery_key>

Using uvx

1. Install uv:

# macOS and Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

2. Use the following configuration:

{
  "mcpServers": {
    "oxylabs": {
      "command": "uvx",
      "args": ["oxylabs-mcp"],
      "env": {
        "OXYLABS_USERNAME": "OXYLABS_USERNAME",
        "OXYLABS_PASSWORD": "OXYLABS_PASSWORD",
        "OXYLABS_AI_STUDIO_API_KEY": "OXYLABS_AI_STUDIO_API_KEY"
      }
    }
  }
}

Manual Setup with Claude Desktop

Navigate to Claude → Settings → Developer → Edit Config and add one of the configurations to the claude_desktop_config.json file.

Manual Setup with Cursor AI

Navigate to Cursor → Settings → Cursor Settings → MCP. Click Add new global MCP server and add one of the configurations.

Environment Variables

Name	Description	Default
OXYLABS_USERNAME	Your Oxylabs Web Scraper API username
OXYLABS_PASSWORD	Your Oxylabs Web Scraper API password
OXYLABS_AI_STUDIO_API_KEY	Your Oxylabs AI Studio API key
LOG_LEVEL	Log level for the logs returned to the client	INFO

Note: At least one set of credentials (Web Scraper API or AI Studio) is required.

Example Queries

When you've set up the MCP server with Claude, you can make requests like:

Web Scraper API Examples

• Could you scrape https://www.google.com/search?q=ai page?
• Scrape https://www.amazon.de/-/en/Smartphone-Contract-Function-Manufacturer-Exclusive/dp/B0CNKD651V with parse enabled
• Use web unblocker with render to scrape https://www.bestbuy.com/site/top-deals/all-electronics-on-sale/pcmcat1674241939957.c

AI Studio Examples

• Use AI scraper to get top news headlines from https://news-site.com in JSON format
• Use AI crawler with prompt "extract all product information" to crawl https://example-store.com
• Use browser agent with task "log in and extract dashboard data" on https://complex-app.com

API Parameters

The Oxylabs MCP Universal Scraper accepts these parameters:

Parameter	Description	Values
url	The URL to scrape	Any valid URL
render	Use headless browser rendering	html or None
geo_location	Sets the proxy's geo location	Brasil, Canada, etc.
user_agent_type	Device type and browser	desktop, tablet, etc.
output_format	The format of the output	links, md, html

Logging

The server provides additional information about tool calls in notification/message events:

{
  "method": "notifications/message",
  "params": {
    "level": "info",
    "data": "Create job with params: {\"url\": \"https://ip.oxylabs.io\"}"
  }
}

How to install this MCP server

For Claude Code

To add this MCP server to Claude Code, run this command in your terminal:

claude mcp add-json "oxylabs" '{"command":"uvx","args":["oxylabs-mcp"],"env":{"OXYLABS_USERNAME":"OXYLABS_USERNAME","OXYLABS_PASSWORD":"OXYLABS_PASSWORD"}}'

See the official Claude Code MCP documentation for more details.

For Cursor

There are two ways to add an MCP server to Cursor. The most common way is to add the server globally in the ~/.cursor/mcp.json file so that it is available in all of your projects.

If you only need the server in a single project, you can add it to the project instead by creating or adding it to the .cursor/mcp.json file.

Adding an MCP server to Cursor globally

To add a global MCP server go to Cursor Settings > Tools & Integrations and click "New MCP Server".

When you click that button the ~/.cursor/mcp.json file will be opened and you can add your server like this:

{
    "mcpServers": {
        "oxylabs": {
            "command": "uvx",
            "args": [
                "oxylabs-mcp"
            ],
            "env": {
                "OXYLABS_USERNAME": "OXYLABS_USERNAME",
                "OXYLABS_PASSWORD": "OXYLABS_PASSWORD"
            }
        }
    }
}

Adding an MCP server to a project

To add an MCP server to a project you can create a new .cursor/mcp.json file or add it to the existing one. This will look exactly the same as the global MCP server example above.

How to use the MCP server

Once the server is installed, you might need to head back to Settings > MCP and click the refresh button.

The Cursor agent will then be able to see the available tools the added MCP server has available and will call them when it needs to.

You can also explicitly ask the agent to use the tool by mentioning the tool name and describing what the function does.

For Claude Desktop

To add this MCP server to Claude Desktop:

1. Find your configuration file:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
• Windows: %APPDATA%\Claude\claude_desktop_config.json
• Linux: ~/.config/Claude/claude_desktop_config.json

2. Add this to your configuration file:

{
    "mcpServers": {
        "oxylabs": {
            "command": "uvx",
            "args": [
                "oxylabs-mcp"
            ],
            "env": {
                "OXYLABS_USERNAME": "OXYLABS_USERNAME",
                "OXYLABS_PASSWORD": "OXYLABS_PASSWORD"
            }
        }
    }
}

3. Restart Claude Desktop for the changes to take effect