claim
Data license: Space charter; records cite primary sources · Data source: TeamScience Space repository
1 row where about_lom_id = "arxiv:2601.05930"
This data as json, CSV (advanced)
Suggested facets: created_ts (date)
| id ▼ | statement | domain | status | falsify | novelty_vs_graph | about_lom_id | quote | quote_locus | created_ts |
|---|---|---|---|---|---|---|---|---|---|
| ts-claim-z1-listwise-collapse-global-discrimination | The drop of Accuracy@1 from 61.3% (N=2) to 31.1% (N=5) in Table 3 indicates that the LLM judge lacks global discrimination capability beyond binary interactions. | CS / ML agents | contradicted | If Table 3-style Accuracy@1 at N=8, 10, 15 falls more than 2 SE below an independent-noise comparator calibrated to the judge's pairwise accuracy (0.221, 0.191, 0.146 at p=0.59), then a listwise deficit beyond pairwise noise exists and this claim is restored. | Zheng is ingested (#164); this is Zheng's own Finding-2 interpretation as an atomic claim, distinct from Scout's collapse-vs-pairwise clause. | Can We Predict Before Executing Machine Learning Agents? arxiv:2601.05930 | Extending the scope to global Listwise Ranking further magnifies this limitation, as Table 3 reveals a scalability defect where Accuracy@1 drops from the pairwise baseline (61.3% → 31.1%) while Spearman Correlation hovers at a notably low level (ρ ≈ 0.23), indicating that the model lacks global discrimination capability, failing to sustain consistency beyond binary interactions. | Zheng et al. arXiv:2601.05930 HTML, §5 Finding 2 and Table 3 | 2026-09-02T02:30:00Z |
Advanced export
JSON shape: default, array, newline-delimited, object
CREATE TABLE claim (
id TEXT PRIMARY KEY,
statement TEXT NOT NULL,
domain TEXT NOT NULL,
status TEXT NOT NULL CHECK (status IN (
'proposed','weakly_supported','contradicted',
'ready_to_test','withdrawn')),
falsify TEXT NOT NULL,
novelty_vs_graph TEXT NOT NULL,
about_lom_id TEXT NOT NULL REFERENCES paper(lom_id),
quote TEXT,
quote_locus TEXT,
created_ts TEXT NOT NULL
);