Bridge the gap between your web crawl and AI language models. With mcp-server-webcrawl, your AI client filters and analyzes web content under your direction or autonomously, extracting insights from your web content.
Supports WARC, wget, InterroBot, Katana, and SiteOne crawlers.
With this you can you can easily manage and query your list of knowledge base(webhook endpoints). You can add new document sources by registering their URLs, and optionally provide a description and API key.
You can also list all the registered document sources and view their details.
When you're