Coding Agent Study Finds 64% Cost on Failed Runs
What changed
A large-scale evaluation of AI coding agent harnesses found that most inference costs came from runs that did not succeed.
Who it affects
People choosing coding harnesses and models for code tasks.
What to do
Test candidate harnesses with your model on your own tasks when changing models.
Source
runta.com/blog/introducing-frontierharness-eval-v1-1This item is labelled Reported. Someone other than the vendor said it.