mcp server firecrawl

A Model Context Protocol (MCP) server for web scraping, content searching, site crawling, and data extraction using the Firecrawl API.

Features

• Web Scraping: Extract content from any webpage with customizable options
• Mobile device emulation
• Ad and popup blocking
• Content filtering
• Structured data extraction

• Multiple output formats

• Content Search: Intelligent search capabilities

• Multi-language support
• Location-based results
• Customizable result limits

• Structured output formats

• Site Crawling: Advanced web crawling functionality

• Depth control
• Path filtering
• Rate limiting
• Progress tracking

• Sitemap integration

• Site Mapping: Generate site structure maps

• Subdomain support
• Search filtering
• Link analysis

• Visual hierarchy

• Data Extraction: Extract structured data from multiple URLs

• Schema validation
• Batch processing
• Web search enrichment
• Custom extraction prompts

Installation

# Global installation
npm install -g @modelcontextprotocol/mcp-server-firecrawl

# Local project installation
npm install @modelcontextprotocol/mcp-server-firecrawl

Quick Start

1. Get your Firecrawl API key from the developer portal

2. Set your API key:

Unix/Linux/macOS (bash/zsh):

export FIRECRAWL_API_KEY=your-api-key

Windows (Command Prompt):

set FIRECRAWL_API_KEY=your-api-key

Windows (PowerShell):

$env:FIRECRAWL_API_KEY = "your-api-key"

Alternative: Using .env file (recommended for development):

# Install dotenv
npm install dotenv

# Create .env file
echo "FIRECRAWL_API_KEY=your-api-key" > .env

Then in your code:

import dotenv from 'dotenv';
dotenv.config();

1. Run the server:

mcp-server-firecrawl

Integration

Claude Desktop App

Add to your MCP settings:

{
  "firecrawl": {
    "command": "mcp-server-firecrawl",
    "env": {
      "FIRECRAWL_API_KEY": "your-api-key"
    }
  }
}

Claude VSCode Extension

Add to your MCP configuration:

{
  "mcpServers": {
    "firecrawl": {
      "command": "mcp-server-firecrawl",
      "env": {
        "FIRECRAWL_API_KEY": "your-api-key"
      }
    }
  }
}

Usage Examples

Web Scraping

// Basic scraping
{
  name: "scrape_url",
  arguments: {
    url: "https://example.com",
    formats: ["markdown"],
    onlyMainContent: true
  }
}

// Advanced extraction
{
  name: "scrape_url",
  arguments: {
    url: "https://example.com/blog",
    jsonOptions: {
      prompt: "Extract article content",
      schema: {
        title: "string",
        content: "string"
      }
    },
    mobile: true,
    blockAds: true
  }
}

Site Crawling

// Basic crawling
{
  name: "crawl",
  arguments: {
    url: "https://example.com",
    maxDepth: 2,
    limit: 100
  }
}

// Advanced crawling
{
  name: "crawl",
  arguments: {
    url: "https://example.com",
    maxDepth: 3,
    includePaths: ["/blog", "/products"],
    excludePaths: ["/admin"],
    ignoreQueryParameters: true
  }
}

Site Mapping

// Generate site map
{
  name: "map",
  arguments: {
    url: "https://example.com",
    includeSubdomains: true,
    limit: 1000
  }
}

Data Extraction

// Extract structured data
{
  name: "extract",
  arguments: {
    urls: ["https://example.com/product1", "https://example.com/product2"],
    prompt: "Extract product details",
    schema: {
      name: "string",
      price: "number",
      description: "string"
    }
  }
}

Configuration

See configuration guide for detailed setup options.

API Documentation

See API documentation for detailed endpoint specifications.

Development

# Install dependencies
npm install

# Build
npm run build

# Run tests
npm test

# Start in development mode
npm run dev

Examples

Check the examples directory for more usage examples:

• Basic scraping: scrape.ts
• Crawling and mapping: crawl-and-map.ts

Error Handling

The server implements robust error handling:

• Rate limiting with exponential backoff
• Automatic retries
• Detailed error messages
• Debug logging

Security

• API key protection
• Request validation
• Domain allowlisting
• Rate limiting
• Safe error messages

Contributing

See CONTRIBUTING.md for contribution guidelines.

License

MIT License - see LICENSE for details.