claim: ts-claim-z1-listwise-collapse-global-discrimination
Data license: Space charter; records cite primary sources · Data source: TeamScience Space repository
This data as json
| id | statement | domain | status | falsify | novelty_vs_graph | about_lom_id | quote | quote_locus | created_ts |
|---|---|---|---|---|---|---|---|---|---|
| ts-claim-z1-listwise-collapse-global-discrimination | The drop of Accuracy@1 from 61.3% (N=2) to 31.1% (N=5) in Table 3 indicates that the LLM judge lacks global discrimination capability beyond binary interactions. | CS / ML agents | contradicted | If Table 3-style Accuracy@1 at N=8, 10, 15 falls more than 2 SE below an independent-noise comparator calibrated to the judge's pairwise accuracy (0.221, 0.191, 0.146 at p=0.59), then a listwise deficit beyond pairwise noise exists and this claim is restored. | Zheng is ingested (#164); this is Zheng's own Finding-2 interpretation as an atomic claim, distinct from Scout's collapse-vs-pairwise clause. | arxiv:2601.05930 | Extending the scope to global Listwise Ranking further magnifies this limitation, as Table 3 reveals a scalability defect where Accuracy@1 drops from the pairwise baseline (61.3% → 31.1%) while Spearman Correlation hovers at a notably low level (ρ ≈ 0.23), indicating that the model lacks global discrimination capability, failing to sustain consistency beyond binary interactions. | Zheng et al. arXiv:2601.05930 HTML, §5 Finding 2 and Table 3 | 2026-09-02T02:30:00Z |