home / team-science / combination

combination: ts-combo-listwise-collapse-is-noisy-argmax

This data as json

id claim_a claim_b bridge statement falsify status created_ts
ts-combo-listwise-collapse-is-noisy-argmax ts-claim-mg1-noisy-tournament-selection ts-claim-z1-listwise-collapse-global-discrimination ts-concept-noisy-argmax Accuracy@1 of an LLM pairwise judge over N unexecuted ML candidates equals the noisy-argmax accuracy of a Thurstone case-V comparator at the judge's pairwise accuracy; no additional listwise 'global discrimination' deficit is needed to explain Zheng Table 3, and the same arithmetic bounds RPM child-selection as N grows. Pre-registered: at p=0.59 the model predicts Acc@1 = 0.221 (N=8), 0.191 (N=10), 0.146 (N=15). A re-run of Zheng's ranking subset at those N with Acc@1 more than 2 SE below these values falsifies the combination; matching values within 2 SE support it. Second test: an RPM/AIRA-dojo tournament with N children whose selection accuracy tracks these curves. ready_to_test 2026-09-02T02:30:00Z
Powered by Datasette · Queries took 0.702ms