Memory & evals · Project

DeepEval

DeepEval tests LLM applications with assertions on outputs, components and agent trajectories.

+ Follow

Builder facts

  • Local test runs. Python evaluation tests can run locally with pytest-style assertions or the DeepEval CLI, including in CI. Source ↗
  • Model and cloud access. Local evaluation needs no Confident AI account. Hosted judge metrics require the configured model provider credentials. A Confident AI account enables cloud reports and collaboration. Source ↗
  • Judge models. Metrics that use model judges support configurable evaluation models, including local Ollama models and custom model wrappers. Source ↗
  • Trajectory scoring. Trace-based evaluation can score the ordered model decisions, tool calls and intermediate steps in an agent run. Source ↗

Sources checked

Publisher update ·

DeepEval Python 4.2.4 ↗

Fixed how async observed methods handle the instance argument in span inputs.

Change timeline

0 stories

No verified releases in this timeline yet.