LSD Web Data Extraction MCP server

This MCP server enables powerful web data extraction through LSD (Language for Structured Data). It connects with the LSD platform to leverage dynamic tools defined as "trips" for web scraping and data processing.

Installation and Setup

Authentication

Before using the MCP server, you need to authenticate with your LSD account credentials.

Using a Configuration File

Create a JSON file named .lsd in your home directory with your email and API key:

{
  "user": "<[email protected]>",
  "password": "<api_key>"
}

Using Environment Variables

Alternatively, set these environment variables:

export LSD_USER='[email protected]'
export LSD_PASSWORD='<api_key>'

Note: Ensure these environment variables are accessible from the process path where the MCP client operates.

Installation Methods

From Source

1. Clone the repository:

git clone https://github.com/lsd-so/mcp.git

2. If using Claude desktop, update your claude_desktop_config.json file:

{
  "mcpServers": {
    "lsd": {
      "command": "node",
      "args": [
        "/<path>/<to>/mcp/build/index.js"
      ]
    }
  }
}

Using a Registry

The MCP server is available through Pulse registry at: https://www.pulsemcp.com/servers/lsd-so-internetdata

Usage Examples

Interaction Mode

The MCP server allows interactive data extraction and research through prompts like lsd_research.

Extraction Mode

You can extract structured data from websites which is then available to LLMs like Claude.

Extending Capabilities

Using LSD Trips

For custom data extraction "skills," you can create and publish LSD trips.

What Are Trips?

Trips are published modules consisting of LSD programs for specific data extraction tasks. They can be created through:

1. The Bicycle browser using the "click language"
2. Direct LSD code writing and publishing

Creating Trips in Bicycle Browser

1. Activate the transcriber by clicking the icon in the top right or pressing Command+k (or Ctrl+k)
2. "Pluck" repeating containers and fields of interest
3. Edit aliases as needed
4. Publish from the workbench by filling out trip details

Using TypeScript

For advanced customization, use the internetdata SDK:

yarn create your-internet

Or with npm:

npm create your-internet

How to install this MCP server

For Claude Code

To add this MCP server to Claude Code, run this command in your terminal:

claude mcp add-json "lsd" '{"command":"node","args":["/<path>/<to>/mcp/build/index.js"]}'

See the official Claude Code MCP documentation for more details.

For Cursor

There are two ways to add an MCP server to Cursor. The most common way is to add the server globally in the ~/.cursor/mcp.json file so that it is available in all of your projects.

If you only need the server in a single project, you can add it to the project instead by creating or adding it to the .cursor/mcp.json file.

Adding an MCP server to Cursor globally

To add a global MCP server go to Cursor Settings > Tools & Integrations and click "New MCP Server".

When you click that button the ~/.cursor/mcp.json file will be opened and you can add your server like this:

{
    "mcpServers": {
        "lsd": {
            "command": "node",
            "args": [
                "/<path>/<to>/mcp/build/index.js"
            ]
        }
    }
}

Adding an MCP server to a project

To add an MCP server to a project you can create a new .cursor/mcp.json file or add it to the existing one. This will look exactly the same as the global MCP server example above.

How to use the MCP server

Once the server is installed, you might need to head back to Settings > MCP and click the refresh button.

The Cursor agent will then be able to see the available tools the added MCP server has available and will call them when it needs to.

You can also explicitly ask the agent to use the tool by mentioning the tool name and describing what the function does.

For Claude Desktop

To add this MCP server to Claude Desktop:

1. Find your configuration file:

• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
• Windows: %APPDATA%\Claude\claude_desktop_config.json
• Linux: ~/.config/Claude/claude_desktop_config.json

2. Add this to your configuration file:

{
    "mcpServers": {
        "lsd": {
            "command": "node",
            "args": [
                "/<path>/<to>/mcp/build/index.js"
            ]
        }
    }
}

3. Restart Claude Desktop for the changes to take effect