usescraper mcp server

Local 2025-08-31 23:15:12 0

Browser Automation @tanevanwifferen/usescraper-mcp-server

A TypeScript-based MCP server utilizing the UseScraper API to provide web scraping capabilities, allowing users to extract content from webpages in various formats.

This is a TypeScript-based MCP server that provides web scraping capabilities using the UseScraper API. It exposes a single tool 'scrape' that can extract content from web pages in various formats.

Features

Tools

scrape - Extract content from a webpage
Parameters:
- url (required): The URL of the webpage to scrape
- format (optional): The format to save the content (text, html, markdown). Default: markdown
- advanced_proxy (optional): Use advanced proxy to circumvent bot detection. Default: false
- extract_object (optional): Object specifying data to extract

Installation

Installing via Smithery

To install UseScraper for Claude Desktop automatically via Smithery:

npx -y @smithery/cli install usescraper-server --client claude

Manual Installation

Clone the repository:

git clone https://github.com/your-repo/usescraper-server.git
cd usescraper-server

Install dependencies:
```
npm install
```
Build the server:
```
npm run build
```

Configuration

To use with Claude Desktop, add the server config:

On MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json On Windows: %APPDATA%/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "usescraper-server": {
      "command": "node",
      "args": ["/path/to/usescraper-server/build/index.js"],
      "env": {
        "USESCRAPER_API_KEY": "your-api-key-here"
      }
    }
  }
}

Replace /path/to/usescraper-server with the actual path to the server and your-api-key-here with your UseScraper API key.

Usage

Once configured, you can use the 'scrape' tool through the MCP interface. Example usage:

{
  "name": "scrape",
  "arguments": {
    "url": "https://example.com",
    "format": "markdown"
  }
}

Development

For development with auto-rebuild:

npm run watch

Debugging

Since MCP servers communicate over stdio, debugging can be challenging. We recommend using the MCP Inspector, which is available as a package script:

npm run inspector

The Inspector will provide a URL to access debugging tools in your browser.

[
  {
    "description": "Scrape content from a webpage using UseScraper API",
    "inputSchema": {
      "properties": {
        "advanced_proxy": {
          "description": "Use advanced proxy to circumvent bot detection (default: false)",
          "type": "boolean"
        },
        "extract_object": {
          "description": "Optional object specifying data to extract",
          "type": "object"
        },
        "format": {
          "description": "Format to save crawled page content. Strongly recommended to keep as markdown for optimal AI processing (default: markdown)",
          "enum": [
            "text",
            "html",
            "markdown"
          ],
          "type": "string"
        },
        "url": {
          "description": "URL to scrape",
          "type": "string"
        }
      },
      "required": [
        "url"
      ],
      "type": "object"
    },
    "name": "scrape"
  }
]