Memory & evals · Project

Promptfoo

Promptfoo runs repeatable comparisons of prompts and model providers against configured test cases.

+ Follow

Builder facts

  • Access. The CLI runs locally. Enterprise offers separate managed and on-premises plans. Source ↗
  • Test matrix. A configuration combines prompts, providers and test cases with variable values. Results can be viewed locally or exported as JSON, spreadsheets or HTML. Source ↗
  • Assertions. Assertions include deterministic checks, custom JavaScript and model-graded rubrics. A hosted provider requires the applicable credentials. Source ↗
  • CI integration. Command-line evaluations can run in CI to check prompt regressions on pull requests. Source ↗

Sources checked

Publisher update ·

Promptfoo 0.126.0 ↗

Added GPT-6.2 Astra and Sol provider support through the Responses API by default. String-based derived metrics gained stricter formula validation before tests run.

Change timeline

0 stories

No verified releases in this timeline yet.