home / mcp / multi fetch mcp server
Provides web content fetching and search capabilities for MCP clients, converting HTML to Markdown and returning content in manageable chunks.
Configuration
View docs{
"mcpServers": {
"alexyangjie-mcp-server-multi-fetch": {
"url": "https://api.firecrawl.dev/mcp",
"headers": {
"FIRECRAWL_API_KEY": "YOUR_API_KEY"
}
}
}
}You run a Model Context Protocol server that fetches and processes web content for you. It converts HTML to Markdown, lets you request content from single or multiple URLs, and supports web search to gather relevant results for your LLM workflows.
Start the server locally and connect your MCP client to it. You can fetch the contents of a single web page, fetch multiple pages in parallel, or run a web search to gather results in markdown and link formats. Use the provided tools to extract content in chunks when you need only portions of a page.
# Install the MCP server and Firecrawl SDK
pip install mcp-server-multi-fetch firecrawl-py
# Set your Firecrawl API key (required)
export FIRECRAWL_API_KEY="fc-YOUR_API_KEY"
# Optionally override the Firecrawl API endpoint
export FIRECRAWL_API_URL="https://api.firecrawl.dev"
# or
mcp-server-multi-fetch --api-url https://api.firecrawl.dev
# Start the server (default run via stdio)
mcp-server-multi-fetch
```} ,{You can customize how the server behaves with a few additional options. For example, you can disable robots.txt obedience for certain requests, change the user agent string, or route traffic through a proxy. The following patterns show common customizations you may apply.
By default, requests from models follow a site's robots.txt only when the request originates from a model tool. If you want to ignore robots.txt checks for all requests, add the argument --ignore-robots-txt to the command line.
The server uses a descriptive user-agent string that varies based on the request source. You can customize it by adding --user-agent=YourUserAgent to the command line. For example, you might set a specific agent for model-originated requests.
If you need to route traffic through a proxy, supply a proxy URL with --proxy-url. The server supports HTTP(S) and SOCKS5 proxies. Example:
``
mcp-server-multi-fetch --proxy-url http://127.0.0.1:8080
`
`
mcp-server-multi-fetch --proxy-url socks5://127.0.0.1:1080
``
You can override the Firecrawl SDK endpoint without environment variables using --api-url. This is useful for testing against different Firecrawl deployments. Example:
``
mcp-server-multi-fetch --api-url https://api.firecrawl.dev
``
If you want to inspect the server while running, you can use an MCP inspector tool. For typical setups, run the inspector with the command tailored to your environment to observe how the server handles fetch and search Tool calls.
Keep your API key secure. Do not hard-code credentials in scripts. Use environment variables or secret management where possible, and rotate keys periodically.
Fetches a URL and returns its contents as markdown with an option to limit length and start from a specific index to enable chunked reading.
Fetches multiple URLs concurrently and returns an array of results, enabling parallel content extraction.
Searches the web via the Firecrawl API and returns results in markdown and link formats by default.