Home › Best AI tools › LLM evaluation and observability tools
LLM evaluation and observability tools
23 tools for tracing, testing and monitoring LLM apps and agents. Platforms and open-source projects that record traces of model and tool calls, run evaluations and monitor quality in production. Compare self-hosting options, framework integrations and how usage is metered.
How to read this page. The tools are grouped by category and are not ranked. Pricing labels come from vendors' own pages when we could confirm them. Always verify plans and limits on the vendor's site.
AI Agents & Frameworks (1)
| Tool | What it does | Pricing | Link |
|---|---|---|---|
| AgentOps | Observability and monitoring for AI agents. | See site | Official site |
AI Developer Tools (22)
| Tool | What it does | Pricing | Link |
|---|---|---|---|
| Arize Phoenix | Open-source observability and evaluation tool for LLM apps. | See site | Official site |
| Braintrust | Evaluation and observability platform for AI products. | Freemium | Official site |
| Comet ML | ML experiment tracking, evaluation and model management. | Freemium | Official site |
| Confident AI | LLM evaluation platform built on the open-source DeepEval framework. | Freemium | Official site |
| Evidently AI | Open-source and cloud tools for evaluating, testing and monitoring ML and LLM systems. | See site | Official site |
| Fiddler AI | Monitoring, evaluation and guardrails platform for enterprise ML models, LLM apps and AI agents. | Freemium | Official site |
| Galileo | AI evaluation and observability platform for measuring, monitoring and protecting LLM apps and agents. | See site | Official site |
| Giskard | Testing and red-teaming platform that finds vulnerabilities and quality issues in LLM agents. | See site | Official site |
| Helicone | Open-source LLM observability and gateway platform. | Freemium | Official site |
| HoneyHive | Observability and evaluation platform for testing and monitoring AI agents. | Freemium | Official site |
| Laminar | Open-source observability and evaluation platform for AI agents. | Freemium | Official site |
| Langfuse | Open-source LLM engineering platform for tracing, evaluation and prompt management. | Freemium | Official site |
| LangSmith | Observability, evaluation and deployment platform for LLM applications. | See site | Official site |
| LangWatch | LLM observability, evaluation and agent testing platform. | Freemium | Official site |
| Opik | Open-source platform from Comet for evaluating and monitoring LLM applications. | See site | Official site |
| Patronus AI | AI research lab that studies where models fail and builds training data and simulated environments to improve them. | See site | Official site |
| Promptfoo | Open-source tool for testing and red-teaming prompts, models and agents. | See site | Official site |
| Pydantic Logfire | Observability platform from the Pydantic team for tracing apps and LLM calls, built on OpenTelemetry. | Freemium | Official site |
| Ragas | Open-source toolkit for evaluating retrieval-augmented generation applications. | Free | Official site |
| TruLens | Open-source library for evaluating and tracing LLM apps and agents. | Free | Official site |
| Weave | Weights & Biases toolkit for tracking and evaluating LLM applications. | See site | Official site |
| WhyLabs | AI observability platform for data and model monitoring. | See site | Official site |