mcp llms txt explorer

Local 2025-09-01 00:07:49 0

Developer Tools @thedaviddias/mcp-llms-txt-explorer

A server that helps discover and analyze websites implementing the llms.txt standard, allowing users to check if websites have llms.txt files and list known compliant websites.

A Model Context Protocol server for exploring websites with llms.txt files. This server helps you discover and analyze websites that implement the llms.txt standard.

Features

Resources

Check websites for llms.txt and llms-full.txt files
Parse and validate llms.txt file contents
Access structured data about compliant websites

Tools

check_website - Check if a website has llms.txt files
Takes domain URL as input
Returns file locations and validation status
list_websites - List known websites with llms.txt files
Returns structured data about compliant websites
Supports filtering by file type (llms.txt/llms-full.txt)

Development

Install dependencies:

pnpm install

Build the server:

pnpm run build

For development with auto-rebuild:

pnpm run watch

Installation

Installing via Smithery

To install mcp-llms-txt-explorer for Claude Desktop automatically via Smithery:

npx -y @smithery/cli install @thedaviddias/mcp-llms-txt-explorer --client claude

Installing Manually

To use this server:

# Clone the repository
git clone https://github.com/thedaviddias/mcp-llms-txt-explorer.git
cd mcp-llms-txt-explorer

# Install dependencies
pnpm install

# Build the server
pnpm run build

Configuration with Claude Desktop

To use with Claude Desktop, add the server config:

On MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json On Windows: %APPDATA%/Claude/claude_desktop_config.json

{
  "mcpServers": {
    "llms-txt-explorer": {
      "command": "/path/to/llms-txt-explorer/build/index.js"
    }
  }
}

For npx usage, you can use:

{
  "mcpServers": {
    "llms-txt-explorer": {
      "command": "npx",
      "args": ["-y", "@thedaviddias/mcp-llms-txt-explorer"]
    }
  }
}

Debugging

Since MCP servers communicate over stdio, debugging can be challenging. We recommend using the MCP Inspector, which is available as a package script:

pnpm run inspector

The Inspector will provide a URL to access debugging tools in your browser.

License

This project is licensed under the MIT License—see the LICENSE file for details.

[
  {
    "description": "Check if a website has llms.txt files",
    "inputSchema": {
      "properties": {
        "url": {
          "description": "URL of the website to check",
          "type": "string"
        }
      },
      "required": [
        "url"
      ],
      "type": "object"
    },
    "name": "check_website"
  },
  {
    "description": "List known websites with llms.txt files",
    "inputSchema": {
      "properties": {
        "filter_llms_full_txt": {
          "description": "Only show websites with llms-full.txt",
          "type": "boolean"
        },
        "filter_llms_txt": {
          "description": "Only show websites with llms.txt",
          "type": "boolean"
        }
      },
      "type": "object"
    },
    "name": "list_websites"
  }
]