RAG Anything MCP server for AI agents

The RAG Anything MCP Server is a comprehensive solution for implementing Retrieval-Augmented Generation capabilities across various documents with full multimodal support. This server leverages the raganything library to process documents with text, images, tables, and equations, allowing for sophisticated querying of document collections.

Installation

Prerequisites

Before running the MCP server, you'll need to set up your environment:

Set your OpenAI API key as an environment variable:

export OPENAI_API_KEY="your-openai-api-key-here"

Install the required dependencies:

uv sync

Start the MCP server:

python main.py

Usage Guide

Processing Documents

Process an Entire Directory

The most common starting point is to process a directory containing multiple documents:

process_directory(
  directory_path="/path/to/documents",
  api_key="your-openai-api-key"
)

Advanced Directory Processing

For more control over the processing, you can customize various parameters:

process_directory(
  directory_path="/path/to/research_papers",
  api_key="your-openai-api-key",
  file_extensions=[".pdf", ".docx"],
  enable_image_processing=True,
  enable_table_processing=True,
  max_workers=6
)

Process a Single Document

If you only need to process one file:

process_single_document(
  file_path="/path/to/important_paper.pdf",
  api_key="your-openai-api-key",
  enable_image_processing=True
)

Querying Documents

Basic Text Query

Once documents are processed, you can query them with natural language:

query_directory(
  directory_path="/path/to/documents",
  query="What are the main findings in these research papers?",
  mode="hybrid"
)

Multimodal Query

For more complex queries that include tables or other structured data:

query_with_multimodal_content(
  directory_path="/path/to/documents",
  query="Compare these results with the document findings",
  multimodal_content=[{
    "type": "table",
    "table_data": "Method,Accuracy,Speed\nRAGAnything,95.2%,120ms\nBaseline,87.3%,180ms",
    "table_caption": "Performance comparison"
  }],
  mode="hybrid"
)

Managing Processed Directories

List Available Directories

To see which directories have been processed and are ready for querying:

list_processed_directories()

Get RAG Information

To retrieve detailed information about a processed directory:

get_rag_info(directory_path="/path/to/documents")

Query Modes

The server supports various query modes for different use cases:

hybrid: Combines local and global search (recommended for most cases)
local: Focuses on local context and entity relationships
global: Provides broader, document-level insights and summaries
naive: Simple keyword-based search without graph reasoning
mix: Combines multiple approaches for comprehensive results
bypass: Direct access without RAG processing

Multimodal Content Support

The server can process and analyze several types of content:

Images: Automatic caption generation and visual analysis
Tables: Structure extraction and content analysis
Equations: LaTeX parsing and mathematical reasoning
Charts/Graphs: Visual data interpretation

Example Multimodal Content Format

When using multimodal queries, content should be formatted as shown:

[
  {
    "type": "table",
    "table_data": "Method,Accuracy\nRAGAnything,95.2%\nBaseline,87.3%",
    "table_caption": "Performance comparison"
  },
  {
    "type": "equation",
    "latex": "P(d|q) = \\frac{P(q|d) \\cdot P(d)}{P(q)}",
    "equation_caption": "Document relevance probability"
  }
]

Supported File Types

The MCP server can process various document formats:

PDF documents (.pdf)
Microsoft Word (.docx)
PowerPoint presentations (.pptx)
Text files (.txt)
Markdown files (.md)

Performance Considerations

Use the max_workers parameter to control parallel processing based on your system capabilities
Image and multimodal processing requires more computational resources
Vision processing using GPT-4o is more expensive than regular text processing
Processed documents are stored locally, allowing for efficient repeated queries

How to install this MCP server

For Claude Code

To add this MCP server to Claude Code, run this command in your terminal:

claude mcp add-json "rag-anything" '{"command":"python","args":["main.py"]}'

See the official Claude Code MCP documentation for more details.

For Cursor

There are two ways to add an MCP server to Cursor. The most common way is to add the server globally in the ~/.cursor/mcp.json file so that it is available in all of your projects.

If you only need the server in a single project, you can add it to the project instead by creating or adding it to the .cursor/mcp.json file.

Adding an MCP server to Cursor globally

To add a global MCP server go to Cursor Settings > Tools & Integrations and click "New MCP Server".

When you click that button the ~/.cursor/mcp.json file will be opened and you can add your server like this:

{
    "mcpServers": {
        "rag-anything": {
            "command": "python",
            "args": [
                "main.py"
            ]
        }
    }
}

Adding an MCP server to a project

To add an MCP server to a project you can create a new .cursor/mcp.json file or add it to the existing one. This will look exactly the same as the global MCP server example above.

How to use the MCP server

Once the server is installed, you might need to head back to Settings > MCP and click the refresh button.

The Cursor agent will then be able to see the available tools the added MCP server has available and will call them when it needs to.

You can also explicitly ask the agent to use the tool by mentioning the tool name and describing what the function does.

For Claude Desktop

To add this MCP server to Claude Desktop:

1. Find your configuration file:

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%\Claude\claude_desktop_config.json
Linux: ~/.config/Claude/claude_desktop_config.json

2. Add this to your configuration file:

{
    "mcpServers": {
        "rag-anything": {
            "command": "python",
            "args": [
                "main.py"
            ]
        }
    }
}

3. Restart Claude Desktop for the changes to take effect