sadiuysal/crawl4ai-mcp-server
89 stars · Last commit 2026-02-06
🕷️ A lightweight Model Context Protocol (MCP) server that exposes Crawl4AI web scraping and crawling capabilities as tools for AI agents. Similar to Firecrawl's API but self-hosted and free. Perfect for integrating web scraping into your AI workflows with OpenAI Agents SDK, Cursor, Claude Code, and other MCP-compatible tools.
README preview
# Crawl4AI MCP Server 🕷️ **A lightweight Model Context Protocol (MCP) server that exposes [Crawl4AI](https://docs.crawl4ai.com/) web scraping and crawling capabilities as tools for AI agents.** Similar to Firecrawl's API but self-hosted and free. Perfect for integrating web scraping into your AI workflows with OpenAI Agents SDK, Cursor, Claude Code, and other MCP-compatible tools. ## Features - **🔧 MCP Tools**: Exposes 4 powerful tools: `scrape`, `crawl`, `crawl_site`, `crawl_sitemap` via stdio MCP server - **🌐 Web Scraping**: Single-page scraping with markdown extraction - **🕷️ Web Crawling**: Multi-page breadth-first crawling with depth control - **🧠 Adaptive Crawling**: Smart crawling that stops when sufficient content is gathered - **🛡️ Safety**: Blocks internal networks, localhost, and private IPs - **📱 Agent Ready**: Works with OpenAI Agents SDK, Cursor, and Claude Code - **⚡ Fast**: Powered by Playwright and Crawl4AI's optimized extraction ## 🚀 Quick Start Choose between **Docker** (recommended) or **manual installation**: