llm-evaluation
Comprehensive framework for evaluating LLM performance using automated metrics, human review, and LLM-as-Judge.
Install
npx skills add https://github.com/wshobson/agents --skill llm-evaluationStats
| Total installs | 3,892 |
| Weekly installs | 3.9K |
| GitHub stars | 32.4K |
| First seen | Jan 20, 2026 |
| Source | @wshobson/agents |
Summary
- Covers three evaluation approaches: automated metrics (BLEU, ROUGE, BERTScore, accuracy, precision/recall), human evaluation across dimensions like accuracy and coherence, and LLM-as-Judge for pointwise, pairwise, and reference-based scoring
- Includes implementations for text generation, classification, and retrieval (RAG) evaluation with ready-to-use metric functions and custom metric support
- Provides A/B testing framework with statistical significance testing, effect size calculation, and regression detection to catch performance drops before deployment
- Integrates with LangSmith for dataset management and experiment tracking, plus benchmarking utilities for tracking progress over time
Tags
Related skills
| Skill | Installs | vs llm-evaluation |
|---|---|---|
| prd-to-issues | 4,084 | +192 |
| command development | 3,853 | -39 |
| elevenlabs-dialogue | 3,673 | -219 |
| skill development | 5,444 | +1,552 |
| ai-elements | 5,222 | +1,330 |
FAQ
- How many installs does llm-evaluation have?
- llm-evaluation has 3,892 total installs and 3.9K installs this week.
- Where is llm-evaluation hosted?
- llm-evaluation is published by @wshobson/agents at https://github.com/wshobson/agents.
- How many GitHub stars does llm-evaluation have?
- llm-evaluation has 32.4K GitHub stars.
- When was llm-evaluation first indexed?
- OrangeBot.AI first indexed llm-evaluation on Jan 20, 2026.
- How do I install llm-evaluation?
- Run: npx skills add https://github.com/wshobson/agents --skill llm-evaluation