This MCP server enables Claude to perform object detection, segmentation, classification, and real-time camera analysis using YOLO computer vision models. It allows Claude to see and understand images through state-of-the-art vision capabilities.
Create and navigate to a project directory:
mkdir yolo-mcp-service
cd yolo-mcp-service
Download or clone the project:
# If using git:
git clone https://github.com/GongRzhe/YOLO-MCP-Server.git .
Create a virtual environment:
# Windows
python -m venv .venv
# macOS/Linux
python3 -m venv .venv
Activate the virtual environment:
# Windows
.venv\Scripts\activate
# macOS/Linux
source .venv/bin/activate
Run the setup script:
python setup.py
The script will install dependencies and generate an MCP configuration file with connection details for different MCP clients.
Create a models directory and download some common YOLO models:
# Create models directory
mkdir models
# Download YOLOv8 models
curl -L https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n.pt -o models/yolov8n.pt
curl -L https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n-seg.pt -o models/yolov8n-seg.pt
curl -L https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n-cls.pt -o models/yolov8n-cls.pt
curl -L https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n-pose.pt -o models/yolov8n-pose.pt
For Windows PowerShell:
mkdir models
Invoke-WebRequest -Uri "https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n.pt" -OutFile "models/yolov8n.pt"
Invoke-WebRequest -Uri "https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n-seg.pt" -OutFile "models/yolov8n-seg.pt"
Invoke-WebRequest -Uri "https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n-cls.pt" -OutFile "models/yolov8n-cls.pt"
Invoke-WebRequest -Uri "https://github.com/ultralytics/assets/releases/download/v0.0.0/yolov8n-pose.pt" -OutFile "models/yolov8n-pose.pt"
For Claude web: Set up the service on your local machine and use the configuration provided by the setup script in your MCP client.
For Claude Desktop:
Always start by checking which models are available:
I'd like to use the YOLO tools. Can you first check which models are available on my system?
<function_calls>
<invoke name="list_available_models">
</invoke>
</function_calls>
Analyze an image file on your computer:
Can you analyze this image file for objects?
<function_calls>
<invoke name="analyze_image_from_path">
<parameter name="image_path">/path/to/your/image.jpg</parameter>
<parameter name="confidence">0.3</parameter>
</invoke>
</function_calls>
Specify a particular model:
<function_calls>
<invoke name="analyze_image_from_path">
<parameter name="image_path">/path/to/your/image.jpg</parameter>
<parameter name="model_name">yolov8n.pt</parameter>
<parameter name="confidence">0.4</parameter>
</invoke>
</function_calls>
Perform detailed analysis combining multiple detection methods:
<function_calls>
<invoke name="comprehensive_image_analysis">
<parameter name="image_path">/path/to/your/image.jpg</parameter>
<parameter name="confidence">0.3</parameter>
</invoke>
</function_calls>
Create segmentation masks for identified objects:
<function_calls>
<invoke name="segment_objects">
<parameter name="image_data">/path/to/your/image.jpg</parameter>
<parameter name="is_path">true</parameter>
<parameter name="model_name">yolov8n-seg.pt</parameter>
</invoke>
</function_calls>
Classify the entire image content:
<function_calls>
<invoke name="classify_image">
<parameter name="image_data">/path/to/your/image.jpg</parameter>
<parameter name="is_path">true</parameter>
<parameter name="model_name">yolov8n-cls.pt</parameter>
<parameter name="top_k">5</parameter>
</invoke>
</function_calls>
Start real-time detection:
<function_calls>
<invoke name="start_camera_detection">
<parameter name="model_name">yolov8n.pt</parameter>
<parameter name="confidence">0.3</parameter>
</invoke>
</function_calls>
Get the latest detections:
<function_calls>
<invoke name="get_camera_detections">
</invoke>
</function_calls>
Stop the camera:
<function_calls>
<invoke name="stop_camera_detection">
</invoke>
</function_calls>
<function_calls>
<invoke name="train_model">
<parameter name="dataset_path">/path/to/your/dataset</parameter>
<parameter name="model_name">yolov8n.pt</parameter>
<parameter name="epochs">50</parameter>
</invoke>
</function_calls>
<function_calls>
<invoke name="validate_model">
<parameter name="model_path">/path/to/your/trained/model.pt</parameter>
<parameter name="data_path">/path/to/validation/dataset</parameter>
</invoke>
</function_calls>
<function_calls>
<invoke name="export_model">
<parameter name="model_path">/path/to/your/model.pt</parameter>
<parameter name="format">onnx</parameter>
</invoke>
</function_calls>
Try different camera IDs if camera doesn't work:
<function_calls>
<invoke name="start_camera_detection">
<parameter name="camera_id">1</parameter> <!-- Try 0, 1, or 2 -->
</invoke>
</function_calls>
Check configured model directories:
<function_calls>
<invoke name="get_model_directories">
</invoke>
</function_calls>
For better performance with limited resources, use smaller models (e.g., yolov8n.pt instead of larger variants).
There are two ways to add an MCP server to Cursor. The most common way is to add the server globally in the ~/.cursor/mcp.json file so that it is available in all of your projects.
If you only need the server in a single project, you can add it to the project instead by creating or adding it to the .cursor/mcp.json file.
To add a global MCP server go to Cursor Settings > MCP and click "Add new global MCP server".
When you click that button the ~/.cursor/mcp.json file will be opened and you can add your server like this:
{
"mcpServers": {
"cursor-rules-mcp": {
"command": "npx",
"args": [
"-y",
"cursor-rules-mcp"
]
}
}
}
To add an MCP server to a project you can create a new .cursor/mcp.json file or add it to the existing one. This will look exactly the same as the global MCP server example above.
Once the server is installed, you might need to head back to Settings > MCP and click the refresh button.
The Cursor agent will then be able to see the available tools the added MCP server has available and will call them when it needs to.
You can also explictly ask the agent to use the tool by mentioning the tool name and describing what the function does.