• Балашов Артем Игоревич's avatar
    docs: replace the anecdotal README claim with three real measurements
    · 3f97a852
    Балашов Артем Игоревич создал
    The README advertised "zero bugs shipped over two weeks" on a single user and
    two projects. That was an anecdote that read like a result. It is now replaced
    by what can be shown, including the parts that are unflattering.
    
    Held-out benchmark, PARTIAL: an existing SWE-bench Multilingual harness resolved
    7 of 12 instances actually run (58 of the 70-instance subset produced empty
    patches and were never attempted), all on Sonnet in small-fix mode. No same-model
    single-agent baseline exists, so the figure has nothing to compare against.
    
    Process behaviour over 340 completed tasks in 6 real projects, 235 with a
    parseable stage ledger. Every rate is conditional on the stage returning a
    verdict, and the denominators are published per row: a reviewing stage objected
    in 39.3% (70/178, or 29.8% of the full ledgered corpus), the fixer needed a
    second round in 5.3% (9/169). The loop converges on the first pass almost always,
    though the tail is real ...
    3f97a852