mcp server requests

中文

An MCP server that provides HTTP request capabilities, enabling LLMs to fetch and process web content.

Features

• Supports converting web content to Markdown format
• Supports filtering out content useless for LLMs
• Supports custom User-Agent headers
• Supports random User-Agent headers
• Supports custom request headers in HTTP requests
• Supports full HTTP methods (GET, POST, PUT, DELETE, PATCH)
• LLMs can access complete HTTP response header information

Installation

git clone https://github.com/coucya/mcp-server-requests.git
cd mcp-server-requests
pip install .

Usage

MCP Server Configuration

{
    "mcpServers": {
        "mcp-server-requests": {
            "command": "python",
            "args": [
                "-m",
                "mcp_server_requests"
            ]
        }
    }
}

Command Line

0. Start MCP Server

Start the MCP server directly:

python -m mcp_server_requests

Options

• --user-agent TEXT: Specify custom User-Agent string
• --random-user-agent [browser=xxx;os=xxx]: Use randomly generated User-Agent
• --force-user-agent: Force using command line specified User-Agent, ignoring LLM provided UA
• --list-os-and-browser: List available browsers and OS for random User-Agent generation

Option Details

• --user-agent and --random-user-agent are mutually exclusive and cannot be used together
• User-Agent setup methods:
• Custom string: --user-agent "Mozilla/5.0 (...)"
• Fully random: --random-user-agent

• Conditional random generation:

  • Specify browser type: --random-user-agent browser=chrome
  • Specify OS: --random-user-agent os=windows
  • Both browser and OS: --random-user-agent browser=chrome;os=windows
  • Note: Browser and OS parameters are case insensitive

• Use --list-os-and-browser to view available browsers and OS for --random-user-agent.

• --force-user-agent controls User-Agent priority:

• When enabled: Prioritize command line specified User-Agent (via --user-agent or --random-user-agent), ignoring LLM provided UA
• When disabled:
  • If LLM provides User-Agent, use that
  • Otherwise use command line specified User-Agent

1. fetch - Fetch Web Content

The fetch subcommand is equivalent to the fetch tool functionality, demonstrating fetch capabilities.

python -m mcp_server_requests fetch <URL> [--return-content {raw,basic_clean,strict_clean,markdown}]

Options: - --return-content: Return content type (default: markdown) - raw: Return raw unprocessed HTML content - basic_clean: Basic cleanup, removing non-display tags like script, style - strict_clean: Strict cleanup, removing non-display tags and most HTML attributes - markdown: Convert HTML to clean Markdown format

Example:

python -m mcp_server_requests fetch https://example.com

2. get - Execute HTTP GET Request

The get subcommand is equivalent to the http_get tool functionality, demonstrating http_get capabilities.

python -m mcp_server_requests get <URL> [--headers HEADERS]

Options: - --headers: Custom request headers (format: "key1=value1;key2=value2")

3. post - Execute HTTP POST Request

The post subcommand is equivalent to the http_post tool functionality, demonstrating http_post capabilities.

python -m mcp_server_requests post <URL> [--headers HEADERS] [--data TEXT]

Options: - --headers: Custom request headers - --data: Request body data

4. put - Execute HTTP PUT Request

The put subcommand is equivalent to the http_put tool functionality, demonstrating http_put capabilities.

python -m mcp_server_requests put <URL> [--headers HEADERS] [--data TEXT]

Options: Same as POST method

5. delete - Execute HTTP DELETE Request

The delete subcommand is equivalent to the http_delete tool functionality, demonstrating http_delete capabilities.

python -m mcp_server_requests delete <URL> [--headers HEADERS] [--data TEXT]

Options: Same as POST method

Functionality

Available Tools

1. fetch - Fetch web content

2. Parameters:

   • url (required): Target URL
   • return_content (optional): Return content type ('raw', 'basic_clean', 'strict_clean', 'markdown')
   • raw: Return raw HTML content
   • basic_clean: Return filtered HTML content, removing non-display tags like script, style
   • strict_clean: Return filtered HTML content, removing non-display tags and most useless HTML attributes
   • markdown: Return HTML converted to Markdown

3. http_get - Execute HTTP GET request

4. Parameters:

   • url (required): Target URL
   • query (optional): Query parameter key-value pairs
   • headers (optional): Custom request headers
   • LLM may specify User-Agent in headers, whether to use it is controlled by --force-user-agent (same applies to other tools)

5. http_post - Execute HTTP POST request

6. Parameters:

   • url (required): Target URL
   • query (optional): Query parameter key-value pairs
   • headers (optional): Custom request headers
   • data (optional): Request body data (text)
   • json (optional): Request body data (JSON)
   • data and json cannot be used together

7. http_put - Execute HTTP PUT request

8. Parameters: Same as http_post

9. http_patch - Execute HTTP PATCH request

10. Parameters: Same as http_post

11. http_delete - Execute HTTP DELETE request

12. Parameters: Same as http_post

License

MIT