Provides real-time object detection, segmentation, classification, and analysis via MCP for Claude AI
Configuration
View docs{
"mcpServers": {
"yolo-service": {
"command": "/path/to/.venv/bin/python",
"args": [
"/path/to/server.py"
],
"env": {
"PYTHONPATH": "/path/to"
}
}
}
}You run a YOLO MCP Service that lets Claude AI perform object detection, segmentation, classification, and real-time camera analysis through an MCP (Model Context Protocol). This practical guide walks you through how to use, install, and configure the service so you can analyze images, train models, and integrate with Claude smoothly.
You interact with the service through an MCP client to perform a variety of computer vision tasks. You can check which models are available on your system, run object detection on local images, perform image segmentation or classification, and use your computer’s camera for real-time detection. You can also train, validate, and export models, then test the connection to ensure everything is wired up correctly.
Prerequisites - Python 3.10 or higher - Git (optional, for cloning the repository)
1. Create a project directory and move into it
``
mkdir yolo-mcp-service
cd yolo-mcp-service
``
2. Prepare the project files
``
# If you have the files, copy them to this directory
# If using git:
git clone https://github.com/GongRzhe/YOLO-MCP-Server.git .
``
3. Create a Python virtual environment
``
# On Windows
python -m venv .venv
# On macOS/Linux
python3 -m venv .venv
``
4. Activate the virtual environment
``
# On Windows
.venv\Scripts\activate
# On macOS/Linux
source .venv/bin/activate
``
5. Run the setup to install dependencies and generate MCP config
``
python setup.py
``
6. Review the setup output for MCP configuration details. The setup will show how to run the service and how to integrate with Claude via MCP. The configuration will include the command and arguments to start the server, along with any environment variables that need to be set.
7. Ensure YOLO models are available in the expected directories before running analyses. Create a models directory and place model files inside or download them as shown in the setup instructions. The service will search in the current directory, a models subdirectory, or any directories listed in the CONFIG["model_dirs"] variable in server.py.
8. Configure Claude to connect to the MCP service using the output from the setup. The configuration typically needs to include the MCP server details so Claude can call into the service for the various tool endpoints.
Configuration and environment variables The service may require environment variables to operate correctly. For example, PYTHONPATH may be set to point to the project root so Python can locate modules. Include PYTHONPATH in the server environment as shown in the example MCP configuration.
Troubleshooting If you cannot connect, verify that the MCP command and arguments are correct, ensure the virtual environment is activated, and check that the required model files exist in the configured directories. If the camera fails, try different camera IDs and confirm the application has permission to access hardware.
Security and access When exposing an MCP service locally, ensure your machine is secured and that only trusted clients can initiate requests. Use secure channels if possible and restrict access to the MCP endpoint to prevent unauthorized usage.
This service provides a range of image analysis capabilities that you can invoke through an MCP client. The main tools/functions include: list_available_models, analyze_image_from_path, comprehensive_image_analysis, segment_objects, classify_image, start_camera_detection, get_camera_detections, stop_camera_detection, train_model, validate_model, export_model, and test_connection. Each tool corresponds to a specific operation such as detecting objects in an image, segmenting objects, classifying image content, or controlling real-time camera analysis.
- Start with listing available models to understand what you can run on your system, and then pick a model that matches your task (detection, segmentation, classification, or pose estimation).
- For image analysis, you can provide image paths or base64-encoded data depending on what your client supports. You can also specify a particular model when needed.
- When using the camera for real-time analysis, you can start the camera, fetch detections as needed, and stop the camera when you are finished.
Returns the list of models available on the system so you can choose the appropriate one for your task.
Analyzes an image file at a given path or data source to detect objects, with optional model_name and confidence threshold.
Performs a detailed, combined analysis that may include detection, classification, and other model-driven insights on a single image.
Performs segmentation to identify object boundaries and generate masks for detected objects.
Classifies the overall content of an image using a specified model and returns top results.
Starts real-time object detection using the computer's camera with a chosen model and confidence threshold.
Retrieves the latest detections from the running camera analysis.
Stops the real-time camera detection session.
Trains a custom object detection model on a user-provided dataset with a chosen model architecture.
Evaluates a trained model against a validation dataset to assess performance.
Exports a trained model to a different format such as ONNX.
Checks whether the MCP service is responsive and correctly connected.