SemEval-2020 Task 5

Intra-sentence No Polarity Signal No Strength Signal

SemEval-2020 Task 5 asks systems to recognize counterfactual statements in news text and, for causal ones, extract the antecedent (hypothetical condition) and consequent (hypothetical result) spans — mapped here onto this project's cause/effect roles.

Detection draws from all 20,000 sentences (2,192 causal); extraction and identification draw from a separate, smaller set of 5,501 sentences re-collected specifically for span annotation — about 14% of those have only an antecedent span with no separate consequent, so they contribute a marked entity but no relation.

Every sentence here is counterfactual/irrealis by construction — a distinct modality axis from both polarity and strength that this project's schema doesn't yet have a category for, currently flattened to a plain Relation.Causal with no flag at all.

Overview

Domain	News
Year	2020
Sentences	20,000
Granularity	Intra-sentence

Tasks & Splits

Task	Train	Test
Causality Detection	—	—
Causal Event Candidate Extraction	—	—
Causality Identification	—	—
Causality Extraction	—	—

Citation

@inproceedings{yang:2020,
  title = {{{SemEval-2020 Task}} 5: {{Counterfactual Recognition}}},
  shorttitle = {{{SemEval-2020 Task}} 5},
  booktitle = {Proceedings of the {{Fourteenth Workshop}} on {{Semantic Evaluation}}, {{SemEval}}@{{COLING}} 2020, {{Barcelona}} (Online), {{December}} 12-13, 2020},
  author = {Yang, Xiaoyu and Obadinma, Stephen and Zhao, Huasha and Zhang, Qiong and Matwin, Stan and Zhu, Xiaodan},
  editor = {Herbelot, Aur{\'e}lie and Zhu, Xiaodan and Palmer, Alexis and Schneider, Nathan and May, Jonathan and Shutova, Ekaterina},
  year = 2020,
  pages = {322--335},
  publisher = {International Committee for Computational Linguistics},
  doi = {10.18653/V1/2020.SEMEVAL-1.40},
  url = {https://doi.org/10.18653/v1/2020.semeval-1.40},
  urldate = {2026-01-05}
}