Memory & evals · Project
Promptfoo
Promptfoo runs repeatable comparisons of prompts and model providers against configured test cases.
+ FollowBuilder facts
- Access. The CLI runs locally. Enterprise offers separate managed and on-premises plans. Source ↗
- Test matrix. A configuration combines prompts, providers and test cases with variable values. Results can be viewed locally or exported as JSON, spreadsheets or HTML. Source ↗
- Assertions. Assertions include deterministic checks, custom JavaScript and model-graded rubrics. A hosted provider requires the applicable credentials. Source ↗
- CI integration. Command-line evaluations can run in CI to check prompt regressions on pull requests. Source ↗
Sources checked
Publisher update ·
Promptfoo 0.126.0 ↗Added GPT-6.2 Astra and Sol provider support through the Responses API by default. String-based derived metrics gained stricter formula validation before tests run.
Change timeline
0 storiesNo verified releases in this timeline yet.