F
Firecrawl
Web scraping API built for LLMs — turn any website into LLM-ready data — 89K+ stars
Open Sourcefreemium
About
Firecrawl is a web scraping API that turns entire websites into clean, LLM-ready markdown or structured data. Handles JavaScript rendering, anti-bot bypassing, sitemaps, and recursive crawling. Provides scrape (single URL), crawl (entire site), map (discover URLs), and extract (structured data) endpoints. Essential infrastructure for RAG pipelines and AI agents that need web data.
Details
| Type | web-scraping, data-extraction, api |
| Integrations | langchain, llamaindex, crewai, any-llm |
| Language | python, typescript |
Tags
web-scrapingragapillm-datamarkdownpythonopen-source
Quick Info
- Organization
- Mendable
- Pricing
- freemium
- Free Tier
- Yes
- Updated
- May 26, 2026
124.5Ktoday145
Also in Dev Tools
C
Crawl4AI
Open-source web crawler optimized for LLMs and AI agents — 62K+ stars
OSSFree
unclecode
66.3Ktoday71
H
Headroom Context Optimization
Reduce LLM API costs by 50-90% through advanced context compression
OSSFree
Shubham Saboo
111.8K3d ago78
L
LlamaIndex
Data framework for LLM applications — RAG, agents, and workflows — 47K+ stars
OSSFree
LlamaIndex
49.7K6d ago473