Overview
Crawl4ai
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key.
Open-source web crawler and scraper for LLMs and AI agents: any website into clean, LLM-ready Markdown. Run it yourself, or use Crawl4AI Cloud with one key. 84,259 GitHub stars · Python.
Product summaries may use automated collection and AI assistance. A listing or verification badge is not a hands-on endorsement. Check current features and limits with the provider. How this directory works · Report a correction
84,259 GitHub stars
Written in Python
Topics: ai, ai-agents, crawler, data-extraction
Features & capabilities
Everything it does, in plain English.
Benefits & limitations
What to check in your own trial.
These are listing summaries, not results from an independent hands-on test. Confirm the claims with the provider and try the tool against your own requirements.
✓ Pros
- ✓Active open-source community
- ✓Code is publicly auditable
! Watch-outs
- !Requires manual review before publishing
Who it's for
Where Crawl4ai pays for itself fast.
AI-assisted development
Integration into agentic workflows
Community reviews
Share your take on Crawl4ai
Sign in to leave a review.
Alternatives
Similar tools worth comparing.
ADK Python
An open-source, code-first Python toolkit for building, evaluating, and deploying sophisticated AI agents with flexibility and control.
AgenticSeek
Fully Local Manus AI. No APIs, No $200 monthly bills. Enjoy an autonomous agent that thinks, browses the web, and code for the sole cost of electricity. 🔔 Official updates only via twitter @Martin993886460 (Beware of...

Agent Reach
Give your AI agent eyes to see the entire internet. Read & search Twitter, Reddit, YouTube, GitHub, Bilibili, XiaoHongShu — one CLI, zero API fees.

AI Agent Book
《深入理解 AI Agent:设计原理与工程实践》(李博杰 著)开源主仓库:全书正文、编译版 PDF 与按章配套代码

AI Berkshire
AI 时代的伯克希尔:基于 Claude Code / Codex 的价值投资研究框架。巴菲特·芒格·段永平·李录四大师方法论 + 多Agent并行研究。| AI-era Berkshire: a value investing research framework built for Claude Code / Codex. 4 masters' methodologies + multi-agent adversarial a...

AIHawk
Open-source AI browser agent: an autonomous web agent for web automation in plain language. It browses, clicks, types and reads real websites to do the tasks you describe. Standalone UI, or MCP for Claude Code, Codex ...

