home / skills / openclaw / skills / firecrawler

firecrawler skill

/skills/capt-marbles/firecrawler

This skill scrapes web pages, renders content as markdown, captures screenshots, and extracts data with Firecrawl to fuel informed decisions.

npx playbooks add skill openclaw/skills --skill firecrawler

Review the files below or copy the command above to add this skill to your agents.

Files (3)
SKILL.md
2.7 KB
---
name: firecrawl
description: Web scraping and crawling with Firecrawl API. Fetch webpage content as markdown, take screenshots, extract structured data, search the web, and crawl documentation sites. Use when the user needs to scrape a URL, get current web info, capture a screenshot, extract specific data from pages, or crawl docs for a framework/library.
version: 1.0.0
author: captmarbles
---

# Firecrawl Web Skill

Scrape, search, and crawl the web using [Firecrawl](https://firecrawl.dev).

## Setup

1. Get your API key from [firecrawl.dev/app/api-keys](https://www.firecrawl.dev/app/api-keys)
2. Set the environment variable:
   ```bash
   export FIRECRAWL_API_KEY=fc-your-key-here
   ```
3. Install the SDK:
   ```bash
   pip3 install firecrawl
   ```

## Usage

All commands use the bundled `fc.py` script in this skill's directory.

### Get Page as Markdown

Fetch any URL and convert to clean markdown. Handles JavaScript-rendered content.

```bash
python3 fc.py markdown "https://example.com"
python3 fc.py markdown "https://example.com" --main-only  # skip nav/footer
```

### Take Screenshot

Capture a full-page screenshot of any URL.

```bash
python3 fc.py screenshot "https://example.com" -o screenshot.png
```

### Extract Structured Data

Pull specific fields from a page using a JSON schema.

**Schema example** (`schema.json`):
```json
{
  "type": "object",
  "properties": {
    "title": { "type": "string" },
    "price": { "type": "number" },
    "features": { "type": "array", "items": { "type": "string" } }
  }
}
```

```bash
python3 fc.py extract "https://example.com/product" --schema schema.json
python3 fc.py extract "https://example.com/product" --schema schema.json --prompt "Extract the main product details"
```

### Web Search

Search the web and get content from results (may require paid tier).

```bash
python3 fc.py search "Python 3.13 new features" --limit 5
```

### Crawl Documentation

Crawl an entire documentation site. Great for learning new frameworks.

```bash
python3 fc.py crawl "https://docs.example.com" --limit 30
python3 fc.py crawl "https://docs.example.com" --limit 50 --output ./docs
```

**Note:** Each page costs 1 credit. Set reasonable limits.

### Map Site URLs

Discover all URLs on a website before deciding what to scrape.

```bash
python3 fc.py map "https://example.com" --limit 100
python3 fc.py map "https://example.com" --search "api"
```

## Example Prompts

- *"Scrape https://blog.example.com/post and summarize it"*
- *"Take a screenshot of stripe.com"*
- *"Extract the name, price, and features from this product page"*
- *"Crawl the Astro docs so you can help me build a site"*
- *"Map all the URLs on docs.stripe.com"*

## Pricing

Free tier includes 500 credits. 1 credit = 1 page/screenshot/search query.

Overview

This skill provides web scraping, crawling, and search capabilities powered by the Firecrawl API. It fetches page content as clean Markdown, captures full-page screenshots, extracts structured data via JSON schemas, and can crawl or map documentation sites. Use it to get current web content, automate data extraction, or ingest docs for building knowledge bases.

How this skill works

The skill calls Firecrawl endpoints to load pages (including JavaScript-rendered content), convert HTML to Markdown, and produce screenshots. It accepts JSON schema definitions to extract specific fields, supports site mapping and full-site crawls with configurable limits, and can perform web searches that return result content. Each page, screenshot, or search query consumes one Firecrawl credit.

When to use it

  • You need a clean Markdown version of a live web page, including rendered JavaScript content.
  • You want an automated full-page screenshot for visual verification or documentation.
  • You must extract structured fields (title, price, features) from product pages at scale.
  • You need to crawl documentation for a framework or library to build a knowledge base.
  • You want to discover all URLs on a site before selecting pages to scrape.

Best practices

  • Obtain and set a Firecrawl API key as an environment variable before use.
  • Start with map mode to discover site structure, then narrow crawl limits to save credits.
  • Provide concise JSON schemas and optional prompts for accurate structured extraction.
  • Respect robots.txt and site rate limits; set reasonable crawl limits to avoid blocking.
  • Batch requests and limit concurrency to manage credit usage and avoid throttling.

Example use cases

  • Convert a news article URL to Markdown for summarization and archiving.
  • Capture a product page screenshot and extract name, price, and feature list into JSON.
  • Crawl a framework's docs to create a searchable offline knowledge base for a dev team.
  • Map all URLs on a docs site to identify API pages to prioritize for scraping.
  • Run a targeted web search for recent articles about a technology and retrieve content snippets.

FAQ

How many credits does each action use?

Each page fetch, screenshot, or search query consumes one Firecrawl credit.

Can the skill handle JavaScript-rendered pages?

Yes. Firecrawl renders client-side JavaScript before converting pages to Markdown or extracting data.