home / mcp / pyautogui mcp server

PyAutogui MCP Server

A MCP (Model Context Protocol) server that provides automated GUI testing and control capabilities through PyAutoGUI.

Installation

Add the following to your MCP client configuration file.

Configuration

{
  "mcpServers": {
    "hetaobackend-mcp-pyautogui-server": {
      "command": "uv",
      "args": [
        "--directory",
        "/path/to/mcp-pyautogui-server",
        "run",
        "mcp-pyautogui-server"
      ]
    }
  }
}

You can control and test graphical interfaces remotely using this MCP server powered by PyAutoGUI. It lets you move the mouse, press keys, take screenshots, locate images on screen, and fetch screen information, all through a consistent MCP client workflow. This enables automated GUI testing, end-to-end UI flows, and reproducible interactions across platforms.

How to use

To use the PyAutoGUI MCP server, run it locally and connect to it from an MCP client. You will send commands to move the mouse, click, type text, press hotkeys, take screenshots, and search for images on the screen. The server exposes tools that map directly to GUI actions; your client can orchestrate complex interactions by combining these actions into scripts or test flows.

How to install

Prerequisites you need to meet before installing and running the server:

Python 3.12 or later
Pillow (as a PyAutoGUI dependency)
PyAutoGUI
Any other dependencies will be resolved automatically when installing the package

Install the MCP server package using Python's package manager:

pip install mcp-pyautogui-server

Additional setup steps

If you plan to run the server through MCP tooling, you can start it as a local stdio server or via MCP command line helpers. The following configurations show two common approaches: one for development with a local directory, and one for published deployments.

# Development / Unpublished Servers Configuration
{
  "mcpServers": {
    "mcp-pyautogui-server": {
      "command": "uv",
      "args": [
        "--directory",
        "/path/to/mcp-pyautogui-server",
        "run",
        "mcp-pyautogui-server"
      ]
    }
  }
}

Notes on publishing and running

For published deployments, MCP tooling can start the server with a streamlined command. The example below shows how to run the server with the standard MCP launcher.

{
  "mcpServers": {
    "mcp-pyautogui-server": {
      "command": "uvx",
      "args": [
        "mcp-pyautogui-server"
      ]
    }
  }
}

Available tools

Mouse Control

Move the mouse to specific coordinates, click at the current or a designated position, drag-and-drop, and query the current mouse position.

Keyboard Control

Type text, press individual keys, and perform hotkey combinations to simulate user input.

Screen Operations

Take screenshots, get screen size, locate image templates on the screen, and read pixel colors.