Skip to content

Causality Identification

Given a causal sentence and a pair of candidate event spans, causality identification determines whether the two events stand in a causal relation — and, if so, the direction and polarity of that relation.

The task is typically modeled as classification over marked-up text: entity markers are inserted into the sentence to delimit the two candidate spans, and a classifier is applied to the resulting string.

Data Schema

Each split is stored as a Parquet file with the following columns:

Column Type Description
index str Unique sentence identifier
text str Sentence with entity markers <e1>...</e1>, <e2>...</e2>, ...
relations list[dict] Each dict has relationship (Relation int), first (e.g. "e1"), second (e.g. "e2")

Relation values come from causalatee.data.constants.Relation:

Value Constant Meaning
0 Relation.NoRelation The spans are not causally related
1 Relation.Causal first causes second
2 Relation.Countercausal first countercausally relates to second

Example

Input text:  "<e1>The storm</e1> caused <e2>significant flooding</e2>."

Output relations: [
    {"relationship": 1, "first": "e1", "second": "e2"}   # Causal
]

Input text:  "<e1>Sugar</e1> does not cause <e2>hyperactivity</e2>."

Output relations: [
    {"relationship": 2, "first": "e1", "second": "e2"}   # Countercausal
]

A sentence may contain multiple entity pairs and therefore multiple relation entries.

Datasets

Corpus Sentences Domain Year Granularity Reference Links
AltLex 1,000 Wikipedia 2016 Intra-sentence (Hidey and McKeown 2016)1 📄 🤗
BECauSE 2.0 1,803 News 2017 Intra-sentence (Dunietz et al. 2017)2 📄 🤗
BioCause 851 Medical 2013 Inter-sentence (whole document/section) (Mihaila et al. 2013)3 📄 🤗
Countercausal News Corpus (CCNC) 3,415 News 2025 Intra-sentence (Hagen et al. 2025)4 📄
Causal News Corpus (CNC) 3,248 News 2022 Intra-sentence (Tan et al. 2022)5 🤗
Causal News Corpus V2 / RECESS (CNCv2) 3,415 News 2023 Intra-sentence (Tan et al. 2023)6 📄 🤗
COPA 2,000 General 2011 Intra-sentence (Roemmele et al. 2011)7 🤗
CausalTimeBank (CTB) 2,201 News 2014 Intra-sentence (Mirza et al. 2014)8 📄 🤗
CaTeRS 488 Fiction 2016 Intra-sentence (Mostafazadeh et al. 2016)9 📄 🤗
EventStoryLine (ESL) 2,247 News 2017 Intra-sentence (Caselli and Vossen 2017)10 📄 🤗
EventCausality 583 Web 2011 Intra-sentence (Do et al. 2011)11 🤗
FinCausal 2020 (FinCausal-20) 22,058 Finance 2020 Inter-sentence (whole document/section) (Mariko et al. 2020)12 📄
FinCausal 2023 (FinCausal-23) 2,630 Financial news 2023 Inter-sentence (whole document/section) (Moreno-Sandoval et al. 2023)13 📄
Penn Discourse Treebank 3.0 (PDTB 3.0) — News / WSJ 2019 Intra-sentence (Webber et al. 2019)14 🤗
PolitiCause 17,780 Politics 2024 Intra-sentence (Corral et al. 2024)15 📄
PubMedCausal 30,000 Biomedical 2026 Intra-sentence (Kunle-John et al. 2026)16 📄 🤗
SCITE 5,236 Science 2021 Intra-sentence (Li et al. 2021)17 📄 🤗
SemEval-2007 Task 4 220 Web 2007 Intra-sentence [@girju:2007]
SemEval-2010 Task 8 10,690 General 2010 Intra-sentence (Hendrickx et al. 2010)18 🤗
SemEval-2020 Task 5 20,000 News 2020 Intra-sentence (Yang et al. 2020)19 📄
TCR 172 News 2018 Inter-sentence (whole document/section) (Ning et al. 2018)20 📄 🤗
UniCausal 14,903 Multiple 2023 Intra-sentence (Tan et al. 2023)21 📄

Models

Model Type Reference Links
k-CNN Knowledge- + data-channel CNN for relation classification (Li and Mao 2019)22 📄
SDP Causality Extraction Dependency-based baseline (Girju 2003)23 📄
Sentence Embeddings Fine-tuned pretrained transformer encoder (dense sentence/token embeddings) — —

A model classifying exactly the two spans marked in text implements causalatee.models.PairwiseIdentification — what nearly every trained relation classifier in the literature implements, and what every current identification model in this repo implements. A model that instead classifies every relation among an arbitrary number of marked spans in one call implements the more general causalatee.models.Identification; see lift_pairwise_identification for building one of those from any PairwiseIdentification model. Full interfaces in the API reference.

Models are evaluated using macro-averaged F₁.

Example

See the Fine-tuning for Causality Identification notebook for a step-by-step walkthrough of fine-tuning RoBERTa with entity-marker special tokens using the HuggingFace Trainer.

  1. Hidey, Christopher, and Kathy McKeown. 2016. “Identifying Causal Relations Using Parallel Wikipedia Articles.” Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers. https://doi.org/10.18653/V1/P16-1135. ↩

  2. Dunietz, Jesse, Lori S. Levin, and Jaime G. Carbonell. 2017. “The BECauSE Corpus 2.0: Annotating Causality and Overlapping Relations.” In Proceedings of the 11th Linguistic Annotation Workshop, LAW@EACL 2017, Valencia, Spain, April 3, 2017, edited by Nathan Schneider and Nianwen Xue. Association for Computational Linguistics. https://doi.org/10.18653/V1/W17-0812. ↩

  3. Mihaila, Claudiu, Tomoko Ohta, Sampo Pyysalo, and Sophia Ananiadou. 2013. “BioCause: Annotating and Analysing Causality in the Biomedical Domain.” BMC Bioinform. 14: 2. https://doi.org/10.1186/1471-2105-14-2. ↩

  4. Hagen, Tim, Niklas Deckers, Felix Wolter, Harrisen Scells, and Martin Potthast. 2025. “Investigating Counterclaims in Causality Extraction from Text.” CoRR abs/2510.08224. https://doi.org/10.48550/ARXIV.2510.08224. ↩

  5. Tan, Fiona Anting, Ali Hürriyetoglu, Tommaso Caselli, et al. 2022. “The Causal News Corpus: Annotating Causal Relations in Event Sentences from News.” In Proceedings of the Thirteenth Language Resources and Evaluation Conference, LREC 2022, Marseille, France, 20-25 June 2022, edited by Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, et al. European Language Resources Association. ↩

  6. Tan, Fiona Anting, Hansi Hettiarachchi, Ali Hürriyetoglu, et al. 2023. “RECESS: Resource for Extracting Cause, Effect, and Signal Spans.” In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, IJCNLP 2023 -Volume 1: Long Papers, Nusa Dua, Bali, November 1 - 4, 2023, edited by Jong C. Park, Yuki Arase, Baotian Hu, et al. Association for Computational Linguistics. https://doi.org/10.18653/V1/2023.IJCNLP-MAIN.6. ↩

  7. Roemmele, Melissa, Cosmin Adrian Bejan, and Andrew S. Gordon. 2011. “Choice of Plausible Alternatives: An Evaluation of Commonsense Causal Reasoning.” Logical Formalizations of Commonsense Reasoning, Papers from the 2011 AAAI Spring Symposium, Technical Report SS-11-06, Stanford, California, USA, March 21-23, 2011. ↩

  8. Mirza, Paramita, Rachele Sprugnoli, Sara Tonelli, and Manuela Speranza. 2014. “Annotating Causality in the TempEval-3 Corpus.” In Proceedings of the EACL 2014 Workshop on Computational Approaches to Causality in Language (CAtoCL), edited by Oleksandr Kolomiyets, Marie-Francine Moens, Martha Palmer, James Pustejovsky, and Steven Bethard. Association for Computational Linguistics. https://doi.org/10.3115/v1/W14-0702. ↩

  9. Mostafazadeh, Nasrin, Alyson Grealish, Nathanael Chambers, James F. Allen, and Lucy Vanderwende. 2016. “CaTeRS: Causal and Temporal Relation Scheme for Semantic Annotation of Event Structures.” In Proceedings of the Fourth Workshop on Events, EVENTS@HLT-NAACL 2016, San Diego, California, USA, June 17, 2016, edited by Martha Palmer, Eduard H. Hovy, Teruko Mitamura, and Tim O’Gorman. Association for Computational Linguistics. https://doi.org/10.18653/V1/W16-1007. ↩

  10. Caselli, Tommaso, and Piek Vossen. 2017. “The Event StoryLine Corpus: A New Benchmark for Causal and Temporal Relation Extraction.” In Proceedings of the Events and Stories in the News Workshop@ACL 2017, Vancouver, Canada, August 4, 2017, edited by Tommaso Caselli, Ben Miller, Marieke van Erp, et al. Association for Computational Linguistics. https://doi.org/10.18653/V1/W17-2711. ↩

  11. Do, Quang, Yee Seng Chan, and Dan Roth. 2011. “Minimally Supervised Event Causality Identification.” Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, EMNLP 2011, 27-31 July 2011, John McIntyre Conference Centre, Edinburgh, UK, A Meeting of SIGDAT, a Special Interest Group of the ACL, 294–303. ↩

  12. Mariko, Dominique, Hanna Abi-Akl, Estelle Labidurie, Stephane Durfort, Hugues De Mazancourt, and Mahmoud El-Haj. 2020. “The Financial Document Causality Detection Shared Task (FinCausal 2020).” In Proceedings of the 1st Joint Workshop on Financial Narrative Processing and MultiLing Financial Summarisation, edited by Dr Mahmoud El-Haj, Dr Vasiliki Athanasakou, Dr Sira Ferradans, et al. COLING. https://aclanthology.org/2020.fnp-1.3/. ↩

  13. Moreno-Sandoval, Antonio, Jordi Porta-Zamorano, Blanca Carbajo-Coronado, Doaa Samy, Dominique Mariko, and Mahmoud El-Haj. 2023. “The Financial Document Causality Detection Shared Task (FinCausal 2023).” 2023 IEEE International Conference on Big Data (BigData), 2855–60. https://doi.org/10.1109/BigData59044.2023.10386745. ↩

  14. Webber, Bonnie, Rashmi Prasad, Alan Lee, and Aravind Joshi. 2019. “The Penn Discourse Treebank 3.0 Annotation Manual.” Philadelphia, University of Pennsylvania 35: 108. ↩

  15. Corral, Paulina Garcia, Hanna Béchara, Ran Zhang, and Slava Jankin. 2024. “PolitiCause: An Annotation Scheme and Corpus for Causality in Political Texts.” In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC/COLING 2024, 20-25 May, 2024, Torino, Italy, edited by Nicoletta Calzolari, Min-Yen Kan, Véronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue. ELRA and ICCL. https://aclanthology.org/2024.lrec-main.1124. ↩

  16. Kunle-John, Ifeoluwa, Josiah Paul, Oluwatosin Agbaakin, Peter Aina, Ikenna Odezuligbo, and Sydney Anuyah. 2026. “PubMedCausal: A Span-Level Annotated Corpus for Causal Relation Extraction in Biomedical Text.” CoRR abs/2605.28363. https://doi.org/10.48550/ARXIV.2605.28363. ↩

  17. Li, Zhaoning, Qi Li, Xiaotian Zou, and Jiangtao Ren. 2021. “Causality Extraction Based on Self-Attentive BiLSTM-CRF with Transferred Embeddings.” Neurocomputing 423: 207–19. https://doi.org/10.1016/J.NEUCOM.2020.08.078. ↩

  18. Hendrickx, Iris, Su Nam Kim, Zornitsa Kozareva, et al. 2010. “SemEval-2010 Task 8: Multi-Way Classification of Semantic Relations Between Pairs of Nominals.” In Proceedings of the 5th International Workshop on Semantic Evaluation, SemEval@ACL 2010, Uppsala University, Uppsala, Sweden, July 15-16, 2010, edited by Katrin Erk and Carlo Strapparava. The Association for Computer Linguistics. ↩

  19. Yang, Xiaoyu, Stephen Obadinma, Huasha Zhao, Qiong Zhang, Stan Matwin, and Xiaodan Zhu. 2020. “SemEval-2020 Task 5: Counterfactual Recognition.” In Proceedings of the Fourteenth Workshop on Semantic Evaluation, SemEval@COLING 2020, Barcelona (Online), December 12-13, 2020, edited by Aurélie Herbelot, Xiaodan Zhu, Alexis Palmer, Nathan Schneider, Jonathan May, and Ekaterina Shutova. International Committee for Computational Linguistics. https://doi.org/10.18653/V1/2020.SEMEVAL-1.40. ↩

  20. Ning, Qiang, Zhili Feng, Hao Wu, and Dan Roth. 2018. “Joint Reasoning for Temporal and Causal Relations.” In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long Papers, edited by Iryna Gurevych and Yusuke Miyao. Association for Computational Linguistics. https://doi.org/10.18653/V1/P18-1212. ↩

  21. Tan, Fiona Anting, Xinyu Zuo, and See-Kiong Ng. 2023. “UniCausal: Unified Benchmark and Repository for Causal Text Mining.” In Big Data Analytics and Knowledge Discovery - 25th International Conference, DaWaK 2023, Penang, Malaysia, August 28-30, 2023, Proceedings, edited by Robert Wrembel, Johann Gamper, Gabriele Kotsis, A. Min Tjoa, and Ismail Khalil, vol. 14148. Lecture Notes in Computer Science. Springer. https://doi.org/10.1007/978-3-031-39831-5\23. ↩

  22. Li, Pengfei, and Kezhi Mao. 2019. “Knowledge-Oriented Convolutional Neural Network for Causal Relation Extraction from Natural Language Texts.” Expert Syst. Appl. 115: 512–23. https://doi.org/10.1016/J.ESWA.2018.08.009. ↩

  23. Girju, Roxana. 2003. “Automatic Detection of Causal Relations for Question Answering.” Proceedings of the ACL 2003 Workshop on Multilingual Summarization and Question Answering (Sapporo, Japan), 76–83. https://doi.org/10.3115/1119312.1119322. ↩