Unsloth MCP server for AI agents

Unsloth MCP Server provides a server interface for Unsloth, a library that makes LLM fine-tuning twice as fast while using 80% less memory. This server enables you to optimize, fine-tune, and generate text with various LLMs through a convenient MCP (Model Context Protocol) interface.

Installation

Prerequisites

Python 3.10, 3.11, or 3.12 (not 3.13)
CUDA 11.8 or 12.1+ (recommended)
PyTorch 2.0+
Node.js and npm
NVIDIA GPU with CUDA support (recommended)

Setup Steps

First, install the Unsloth library:

pip install unsloth

Install and build the server:

cd unsloth-server
npm install
npm run build

Add the server to your MCP settings by adding the following configuration:

{
  "mcpServers": {
    "unsloth-server": {
      "command": "node",
      "args": ["/path/to/unsloth-server/build/index.js"],
      "env": {
        "HUGGINGFACE_TOKEN": "your_token_here" // Optional
      },
      "disabled": false,
      "autoApprove": []
    }
  }
}

Basic Usage

Checking Installation

Verify if Unsloth is properly installed on your system:

const result = await use_mcp_tool({
  server_name: "unsloth-server",
  tool_name: "check_installation",
  arguments: {}
});

Listing Supported Models

Get a list of models supported by Unsloth:

const result = await use_mcp_tool({
  server_name: "unsloth-server",
  tool_name: "list_supported_models",
  arguments: {}
});

Loading a Model

Load a pre-trained model with Unsloth optimizations:

const result = await use_mcp_tool({
  server_name: "unsloth-server",
  tool_name: "load_model",
  arguments: {
    model_name: "unsloth/Llama-3.2-1B",
    max_seq_length: 4096,
    load_in_4bit: true
  }
});

Fine-tuning Models

Fine-tune a model using Unsloth's optimizations with LoRA/QLoRA techniques:

const result = await use_mcp_tool({
  server_name: "unsloth-server",
  tool_name: "finetune_model",
  arguments: {
    model_name: "unsloth/Llama-3.2-1B",
    dataset_name: "tatsu-lab/alpaca",
    output_dir: "./fine-tuned-model",
    max_steps: 100,
    batch_size: 2,
    learning_rate: 2e-4
  }
});

Fine-tuning Parameters

model_name: Name of the model to fine-tune
dataset_name: Name of the dataset to use
output_dir: Directory to save the fine-tuned model
max_seq_length: Maximum sequence length (default: 2048)
lora_rank: Rank for LoRA fine-tuning (default: 16)
lora_alpha: Alpha for LoRA fine-tuning (default: 16)
batch_size: Batch size for training (default: 2)
gradient_accumulation_steps: Number of gradient accumulation steps (default: 4)
learning_rate: Learning rate (default: 2e-4)
max_steps: Maximum number of training steps (default: 100)
load_in_4bit: Use 4-bit quantization (default: true)

Generating Text

Generate text using a fine-tuned Unsloth model:

const result = await use_mcp_tool({
  server_name: "unsloth-server",
  tool_name: "generate_text",
  arguments: {
    model_path: "./fine-tuned-model",
    prompt: "Write a short story about a robot learning to paint:",
    max_new_tokens: 512,
    temperature: 0.8
  }
});

Exporting Models

Export a fine-tuned model to various formats for deployment:

const result = await use_mcp_tool({
  server_name: "unsloth-server",
  tool_name: "export_model",
  arguments: {
    model_path: "./fine-tuned-model",
    export_format: "gguf",
    output_path: "./exported-model.gguf",
    quantization_bits: 4
  }
});

Advanced Usage

Using Custom Datasets

You can use custom datasets by formatting them properly:

const result = await use_mcp_tool({
  server_name: "unsloth-server",
  tool_name: "finetune_model",
  arguments: {
    model_name: "unsloth/Llama-3.2-1B",
    dataset_name: "json",
    data_files: {"train": "path/to/your/data.json"},
    output_dir: "./fine-tuned-model"
  }
});

Memory Optimization Tips

For running large models on limited hardware:

Reduce batch size and increase gradient accumulation steps
Use 4-bit quantization (load_in_4bit: true)
Enable gradient checkpointing (use_gradient_checkpointing: true)
Reduce sequence length if possible

Troubleshooting

Common Issues

CUDA Out of Memory: Reduce batch size, use 4-bit quantization, or try a smaller model
Import Errors: Ensure you have correct versions of torch, transformers, and unsloth
Model Not Found: Check that you're using a supported model name

How to add this MCP server to Cursor

There are two ways to add an MCP server to Cursor. The most common way is to add the server globally in the ~/.cursor/mcp.json file so that it is available in all of your projects.

If you only need the server in a single project, you can add it to the project instead by creating or adding it to the .cursor/mcp.json file.

Adding an MCP server to Cursor globally

To add a global MCP server go to Cursor Settings > MCP and click "Add new global MCP server".

When you click that button the ~/.cursor/mcp.json file will be opened and you can add your server like this:

{
    "mcpServers": {
        "cursor-rules-mcp": {
            "command": "npx",
            "args": [
                "-y",
                "cursor-rules-mcp"
            ]
        }
    }
}

Adding an MCP server to a project

To add an MCP server to a project you can create a new .cursor/mcp.json file or add it to the existing one. This will look exactly the same as the global MCP server example above.

How to use the MCP server

Once the server is installed, you might need to head back to Settings > MCP and click the refresh button.

The Cursor agent will then be able to see the available tools the added MCP server has available and will call them when it needs to.

You can also explictly ask the agent to use the tool by mentioning the tool name and describing what the function does.