Scenic GUI Automation MCP server

Scenic MCP is a Model Context Protocol server that enables external keyboard and mouse input control for Scenic GUI applications. It allows tools like Claude Desktop to interact with your Scenic application by sending text, keyboard commands, and mouse movements through a TCP-based connection.

Installation

Add to Your Scenic Application

Add the dependency to your Scenic application's mix.exs:

defp deps do
  [
    {:scenic_mcp, path: "../scenic_mcp"}
  ]
end

Configure Your Scenic Viewport

For scenic_mcp to work correctly, your Scenic application MUST name both the viewport and driver:

def scenic_config() do
  [
    name: :main_viewport,  # Required for viewport lookup
    size: @default_resolution,
    default_scene: {YourApp.RootScene, []},
    drivers: [
      [
        name: :scenic_driver,  # Required for driver lookup
        module: Scenic.Driver.Local,
        window: [
          title: "Your App",
          resizeable: true
        ],
        on_close: :stop_system
      ]
    ]
  ]
end

Install Node.js Dependencies

cd scenic_mcp
npm install
npm run build

Configure for Claude Code

# Add the MCP server to Claude Code
claude mcp add scenic-mcp /path/to/scenic_mcp/dist/index.js

# Verify it was added
claude mcp list

Usage

Using with Claude Code

Once configured, you can use the MCP tools directly within Claude Code conversations:

1. Start your Scenic application (with the ScenicMcp.Server running on port 9999)
2. Use MCP tools in Claude Code:
   • connect_scenic - Test connection to your Scenic app
   • get_scenic_status - Check connection status
   • send_keys - Send keyboard input
   • send_mouse_move - Move mouse cursor
   • send_mouse_click - Click at coordinates
   • inspect_viewport - Get visual description of current screen

Example conversation:

You: "Use the connect_scenic tool to test connection to my Flamelex app"
Claude: [Uses connect_scenic tool and shows connection status]

You: "Send the text 'hello world' using send_keys"
Claude: [Uses send_keys tool to type text into your app]

Available MCP Tools

connect_scenic

Test connection to the Scenic application.

get_scenic_status

Check server status and available commands.

send_keys

Send keyboard input to the Scenic application.

Parameters:

• text (string): Text to type (each character sent as individual key press)
• key (string): Special key name (enter, escape, tab, backspace, delete, up, down, left, right, home, end, page_up, page_down, f1-f12)
• modifiers (array): Modifier keys (ctrl, shift, alt, cmd, meta)

send_mouse_move

Move mouse cursor to specific coordinates.

Parameters:

• x (number): X coordinate
• y (number): Y coordinate

send_mouse_click

Click mouse at specific coordinates.

Parameters:

• x (number): X coordinate
• y (number): Y coordinate
• button (string): Mouse button (left, right, middle) - default: left

get_scenic_graph

Return the script table for a ViewPort, providing a visual description of the scene.

take_screenshot

Capture a screenshot of the current Scenic display.

Parameters:

• filename (string, optional): Custom filename for the screenshot
• format (string, optional): Output format - "path" (default) or "base64"

Example Commands

Send text:

{
  "action": "send_keys",
  "text": "hello world"
}

Send special key:

{
  "action": "send_keys", 
  "key": "enter"
}

Send key with modifiers:

{
  "action": "send_keys",
  "key": "c",
  "modifiers": ["ctrl"]
}

Move mouse:

{
  "action": "send_mouse_move",
  "x": 100,
  "y": 200
}

Click mouse:

{
  "action": "send_mouse_click",
  "x": 150,
  "y": 250,
  "button": "left"
}

Get visual feedback:

{
  "action": "get_scenic_graph"
}

Architecture Overview

The system works through these connected components:

• MCP Client (Claude Desktop) connects to the TypeScript MCP Server
• TypeScript server communicates via TCP port 9999 with the Elixir GenServer Bridge
• The bridge uses ScenicMcp.Probes to interface with Scenic
• Input is sent through Scenic.Driver.send_input/2 to your application

This architecture ensures proper input routing and low-latency communication between the MCP client and your Scenic application.

How to install this MCP server

For Claude Code

To add this MCP server to Claude Code, run this command in your terminal:

claude mcp add-json "scenic-mcp-experimental" '{"command":"npx","args":["-y","scenic-contrib/scenic_mcp_experimental"]}'

See the official Claude Code MCP documentation for more details.

For Cursor

There are two ways to add an MCP server to Cursor. The most common way is to add the server globally in the ~/.cursor/mcp.json file so that it is available in all of your projects.

If you only need the server in a single project, you can add it to the project instead by creating or adding it to the .cursor/mcp.json file.

Adding an MCP server to Cursor globally

To add a global MCP server go to Cursor Settings > Tools & Integrations and click "New MCP Server".

When you click that button the ~/.cursor/mcp.json file will be opened and you can add your server like this:

{
    "mcpServers": {
        "scenic-mcp-experimental": {
            "command": "npx",
            "args": [
                "-y",
                "scenic-contrib/scenic_mcp_experimental"
            ]
        }
    }
}

Adding an MCP server to a project

To add an MCP server to a project you can create a new .cursor/mcp.json file or add it to the existing one. This will look exactly the same as the global MCP server example above.

How to use the MCP server

Once the server is installed, you might need to head back to Settings > MCP and click the refresh button.

The Cursor agent will then be able to see the available tools the added MCP server has available and will call them when it needs to.

You can also explicitly ask the agent to use the tool by mentioning the tool name and describing what the function does.

For Claude Desktop

To add this MCP server to Claude Desktop:

1. Find your configuration file:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
• Windows: %APPDATA%\Claude\claude_desktop_config.json
• Linux: ~/.config/Claude/claude_desktop_config.json

2. Add this to your configuration file:

{
    "mcpServers": {
        "scenic-mcp-experimental": {
            "command": "npx",
            "args": [
                "-y",
                "scenic-contrib/scenic_mcp_experimental"
            ]
        }
    }
}

3. Restart Claude Desktop for the changes to take effect