Vibe-Eyes MCP server for AI agents

Vibe-Eyes is an MCP server that enables Large Language Models to "see" what's happening in browser-based games and applications by capturing, vectorizing, and transmitting canvas content along with debug information. This creates a seamless bridge between visual application states and LLM-assisted debugging.

Installation

Server Setup

# Clone the repository
git clone https://github.com/monteslu/vibe-eyes.git
cd vibe-eyes

# Install dependencies
npm install

Client Integration

Add the Vibe-Eyes client to your browser application by including the required scripts:

<!-- Include Socket.IO client -->
<script src="https://cdn.socket.io/4.7.4/socket.io.min.js"></script>

<!-- Include Vibe-Eyes client -->
<script src="https://cdn.jsdelivr.net/npm/vibe-eyes-client/dist/index.min.js"></script>

<!-- Initialize the client -->
<script>
  // Initialize with configuration
  const vibeEyes = initializeVibeEyes({
    // WebSocket URL to the Vibe-Eyes server
    serverUrl: 'ws://localhost:8869',
    // Capture interval in milliseconds
    captureDelay: 1000,
    // Start capturing automatically after connection
    autoCapture: true
  });
</script>

Usage

Connecting to LLM Agents

# For Claude Code
claude mcp add

Client Configuration Options

The browser client can be configured with several options:

const vibeEyes = initializeVibeEyes({
  serverUrl: 'ws://localhost:8869',
  captureDelay: 1000, // ms between captures
  maxLogs: 10,        // Max console.log entries to store
  maxErrors: 10,      // Max console.error entries to store
  autoCapture: true   // Start capturing automatically
});

Manual Control

You can manually control the capture process:

// Start automatic capturing
vibeEyes.startCaptureLoop();

// Stop automatic capturing
vibeEyes.stopCaptureLoop();

// Trigger one immediate capture
vibeEyes.captureAndSend();

MCP Tool for LLMs

LLMs can access visual and debug information using the MCP tool:

getGameDebug({ includeSvg: true/false })

This tool returns:

Recent console logs and errors from the application
Unhandled exceptions with full stack traces (if any occurred)
Vectorized SVG approximation of the canvas (if includeSvg is true)
Timing information connecting visual state with logs

Example MCP Configuration for Claude

To access Vibe-Eyes from Claude:

{
  "name": "vibe-eyes",
  "url": "http://localhost:8869",
  "tools": [
    {
      "name": "getGameDebug",
      "description": "Retrieves the most recent canvas visualization and debug information from a browser game or application"
    }
  ]
}

Accessing SVG Output Directly

For applications that want to reuse the vectorized SVG output:

WebSocket Response

The server includes the SVG directly in WebSocket responses:

socket.on('debugCapture', (data, callback) => {
  // Response includes:
  // {
  //   success: true,
  //   id: "capture_123",
  //   svg: "<svg>...</svg>",  // Vectorized SVG 
  //   stats: { /* stats data */ }
  // }
});

HTTP Endpoint

Access the latest capture via the /latest endpoint:

fetch('http://localhost:8869/latest')
  .then(res => res.json())
  .then(data => {
    const svg = data.vectorized?.svg;
    // Use the SVG...
  });

Performance Considerations

Configure captureDelay appropriately to balance information frequency with performance
Set autoCapture: false for manual control in performance-sensitive applications
The server processes images sequentially to prevent overload
SVG vectorization balances visual accuracy with size optimization

Standalone Vectorizer CLI

The project includes a standalone CLI tool for vectorizing individual files:

# Install CLI globally
npm install -g vibe-eyes

# Use the CLI
vibe-eyes-vectorize input.png output.svg

# With options
vibe-eyes-vectorize photo.jpg --color-precision 10 --max-iterations 100

How to install this MCP server

For Claude Code

To add this MCP server to Claude Code, run this command in your terminal:

claude mcp add-json "vibe-eyes" '{"url":"http://localhost:8869","tools":[{"name":"getGameDebug","description":"Retrieves the most recent canvas visualization and debug information from a browser game or application"}]}'

See the official Claude Code MCP documentation for more details.

For Cursor

There are two ways to add an MCP server to Cursor. The most common way is to add the server globally in the ~/.cursor/mcp.json file so that it is available in all of your projects.

If you only need the server in a single project, you can add it to the project instead by creating or adding it to the .cursor/mcp.json file.

Adding an MCP server to Cursor globally

To add a global MCP server go to Cursor Settings > Tools & Integrations and click "New MCP Server".

When you click that button the ~/.cursor/mcp.json file will be opened and you can add your server like this:

{
    "mcpServers": {
        "vibe-eyes": {
            "url": "http://localhost:8869",
            "tools": [
                {
                    "name": "getGameDebug",
                    "description": "Retrieves the most recent canvas visualization and debug information from a browser game or application"
                }
            ]
        }
    }
}

Adding an MCP server to a project

To add an MCP server to a project you can create a new .cursor/mcp.json file or add it to the existing one. This will look exactly the same as the global MCP server example above.

How to use the MCP server

Once the server is installed, you might need to head back to Settings > MCP and click the refresh button.

The Cursor agent will then be able to see the available tools the added MCP server has available and will call them when it needs to.

You can also explicitly ask the agent to use the tool by mentioning the tool name and describing what the function does.

For Claude Desktop

To add this MCP server to Claude Desktop:

1. Find your configuration file:

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
Windows: %APPDATA%\Claude\claude_desktop_config.json
Linux: ~/.config/Claude/claude_desktop_config.json

2. Add this to your configuration file:

{
    "mcpServers": {
        "vibe-eyes": {
            "url": "http://localhost:8869",
            "tools": [
                {
                    "name": "getGameDebug",
                    "description": "Retrieves the most recent canvas visualization and debug information from a browser game or application"
                }
            ]
        }
    }
}

3. Restart Claude Desktop for the changes to take effect