tavily-crawl
Crawl websites to extract content as markdown or JSON with semantic filtering for LLM context and research.
Install
npx skills add https://github.com/tavily-ai/skills --skill tavily-crawlStats
| Total installs | 1,671 |
| Weekly installs | 1.7K |
| GitHub stars | 143 |
| First seen | 12 days ago |
| Source | @tavily-ai/skills |
Summary
- Crawl entire site sections with depth and breadth control; filter by path regex, domain, or natural language instructions to focus results
- Save each page as local markdown files via --output-dir, or return structured JSON for agentic processing
- Use semantic instructions with chunk extraction to prevent context bloat when feeding results to LLMs; use full-page extraction for offline documentation downloads
- Supports external link following, image inclusion, timeout configuration, and regex-based path/domain filtering for precise scope control
Tags
Related skills
| Skill | Installs | vs tavily-crawl |
|---|---|---|
| vercel-cli | 1,605 | -66 |
| youtube-downloader | 1,745 | +74 |
| strategic-compact | 1,781 | +110 |
| text-optimizer | 1,860 | +189 |
| trade | 1,698 | +27 |
FAQ
- How many installs does tavily-crawl have?
- tavily-crawl has 1,671 total installs and 1.7K installs this week.
- Where is tavily-crawl hosted?
- tavily-crawl is published by @tavily-ai/skills at https://github.com/tavily-ai/skills.
- How many GitHub stars does tavily-crawl have?
- tavily-crawl has 143 GitHub stars.
- When was tavily-crawl first indexed?
- OrangeBot.AI first indexed tavily-crawl on 12 days ago.
- How do I install tavily-crawl?
- Run: npx skills add https://github.com/tavily-ai/skills --skill tavily-crawl