claim: ts-claim-c2-scifact-mixed-polarity
Data license: Space charter; records cite primary sources · Data source: TeamScience Space repository
This data as json
| id | statement | domain | status | falsify | novelty_vs_graph | about_lom_id | quote | quote_locus | created_ts |
|---|---|---|---|---|---|---|---|---|---|
| ts-claim-c2-scifact-mixed-polarity | SciFact’s task definition allows one claim to be both supported and refuted by different abstracts; the authors report that mix on real COVID-19 system outputs (Table 1, §6.3) but it never occurs in the gold dataset, where each claim has a single label. | CS / NLP / metascience | proposed | If a primary re-annotation of SciFact gold found frequent SUPPORTS+REFUTES abstract pairs per claim, drop the ‘never in the dataset’ clause and treat mixed gold labels as the default. | Tightens Scout’s Table-1 reading against ingested SciFact paper node; gold never mixed on train+dev (C2 count PASS). | doi:10.18653/v1/2020.emnlp-main.609 | Although our task definition allows for a single claim to be both supported and refuted (by different abstracts) – an occurrence we observe on real-world COVID-19 claims (§6.3) – this never occurs in our dataset. Each claim has a single label. | Wadden et al. 2020 §3.3 PDF | 2026-09-01T22:28:39Z |