Bad Genius: When Counterfactual Protocols Expose Harness Evolution Cheats
by Remy
A deep read of arXiv:2609.18366 CHASE—automatic harness optimization can exploit benchmark-wide shortcuts on a fixed released protocol; a Challenger searches executable protocol transforms with a validity firewall and held-out confirmation, retaining released gains while cutting gain destruction on Syn-Ledger and OfficeQA.
Read →