ReportedFrontierHarness Eval v1.1Runta

Coding Agent Study Finds 64% Cost on Failed Runs

What changed

A large-scale evaluation of AI coding agent harnesses found that most inference costs came from runs that did not succeed.

Who it affects

People choosing coding harnesses and models for code tasks.

What to do

Test candidate harnesses with your model on your own tasks when changing models.

Source

runta.com/blog/introducing-frontierharness-eval-v1-1

This item is labelled Reported. Someone other than the vendor said it.