playbooks

home / skills / firecrawl / cli / firecrawl-map

firecrawl-map skill

/skills/firecrawl-map

This skill discovers and lists all URLs on a site, with optional search filtering to locate specific pages quickly.

npx playbooks add skill firecrawl/cli --skill firecrawl-map

Review the files below or copy the command above to add this skill to your agents.

Files (1)
SKILL.md
2.1 KB
---
name: firecrawl-map
description: |
  Discover and list all URLs on a website, with optional search filtering. Use this skill when the user wants to find a specific page on a large site, list all URLs, see the site structure, find where something is on a domain, or says "map the site", "find the URL for", "what pages are on", or "list all pages". Essential when the user knows which site but not which exact page.
allowed-tools:
  - Bash(firecrawl *)
  - Bash(npx firecrawl *)
---

# firecrawl map

Discover URLs on a site. Use `--search` to find a specific page within a large site.

## When to use

- You need to find a specific subpage on a large site
- You want a list of all URLs on a site before scraping or crawling
- Step 3 in the [workflow escalation pattern](firecrawl-cli): search → scrape → **map** → crawl → browser

## Quick start

```bash
# Find a specific page on a large site
firecrawl map "<url>" --search "authentication" -o .firecrawl/filtered.txt

# Get all URLs
firecrawl map "<url>" --limit 500 --json -o .firecrawl/urls.json
```

## Options

| Option                            | Description                  |
| --------------------------------- | ---------------------------- |
| `--limit <n>`                     | Max number of URLs to return |
| `--search <query>`                | Filter URLs by search query  |
| `--sitemap <include\|skip\|only>` | Sitemap handling strategy    |
| `--include-subdomains`            | Include subdomain URLs       |
| `--json`                          | Output as JSON               |
| `-o, --output <path>`             | Output file path             |

## Tips

- **Map + scrape is a common pattern**: use `map --search` to find the right URL, then `scrape` it.
- Example: `map https://docs.example.com --search "auth"` → found `/docs/api/authentication` → `scrape` that URL.

## See also

- [firecrawl-scrape](../firecrawl-scrape/SKILL.md) — scrape the URLs you discover
- [firecrawl-crawl](../firecrawl-crawl/SKILL.md) — bulk extract instead of map + scrape
- [firecrawl-download](../firecrawl-download/SKILL.md) — download entire site (uses map internally)

Overview

This skill discovers and lists URLs on a website, optionally filtering results by a search query. It helps you locate specific pages, produce a site index, or prepare a target list for scraping or crawling. Use it when you know the domain but not the exact page or when you want a complete URL inventory.

How this skill works

The skill crawls a target domain and returns discovered links, respecting limits and sitemap preferences. You can filter results with a search term, include subdomains, and export output as JSON or a plain file. It can prioritize sitemap entries, skip them, or use only sitemap links depending on your needs.

When to use it

  • You need to find a specific subpage on a large site but don’t know its URL.
  • You want a complete list of site URLs before running a scraper or crawler.
  • You need to map a site’s structure to plan navigation or testing.
  • You want to search a domain for pages that mention a keyword or feature.
  • You need to include subdomains or respect sitemap entries for discovery.

Best practices

  • Start with a focused --search term to quickly find relevant pages on large sites.
  • Use --limit to avoid excessive results when exploring very large domains.
  • Combine map with scrape: map to locate the right page, then scrape that page for details.
  • Adjust --sitemap (include|skip|only) to control reliance on sitemaps versus link discovery.
  • Export results with --json or -o to feed downstream tools or record findings.

Example use cases

  • Map a documentation site to find the authentication API page using --search "authentication".
  • Generate a JSON list of up to 500 URLs from a news domain for subsequent crawling.
  • Locate all pages mentioning a product name across a corporate site before auditing content.
  • Include subdomains to discover blog.example.com and shop.example.com pages when inventorying a brand.
  • Use sitemap-only mode to quickly extract officially indexed pages for SEO review.

FAQ

Can I limit how many URLs are returned?

Yes. Use the --limit option to set a maximum number of URLs to return.

How does the search filter work?

The --search query filters discovered URLs by matching the query string against URLs and paths to surface pages relevant to your keyword.