SemEval-2023 Task 8 (RedHOT)

Intra-sentence No Polarity Signal No Strength Signal

SemEval-2023 Task 8 asks systems to identify causal medical claims in Reddit posts and extract related PICO (Population/Intervention/Comparator/Outcome) spans, built on the RedHOT (Reddit Health Online Talk) corpus.

Blocked — no full, directly-fetchable labeled dataset

The official train/test split is distributed only via a CodaLab competition, gated behind shared-task account registration. The full underlying RedHOT corpus requires a self-attested Institutional Review Board (IRB) approval checkbox plus running a script that fetches the real Reddit comment text live via the Reddit API at download time (its own GitHub repo, github.com/sominw/redhot, currently 404s). A small (~178-row) public sample exists on Google Drive in the real annotation schema, but Drive folder listings aren't fetchable via a plain scripted request the way this project's other sources are, and the sample is far too small to be a usable dataset regardless. Not yet attempted — see resources/datasets/SemEval2023T8/conversion_script.py for the full trail of what was checked.

Overview

Domain	Social media / Medical
Year	2023
Sentences	—
Granularity	Intra-sentence

Tasks & Splits

Task	Train	Test

Citation

@inproceedings{khetan:2023,
  title = {{{SemEval-2023 Task}} 8: {{Causal Medical Claim Identification}} and {{Related PIO Frame Extraction}} from {{Social Media Posts}}},
  shorttitle = {{{SemEval-2023 Task}} 8},
  booktitle = {Proceedings of the {{The}} 17th {{International Workshop}} on {{Semantic Evaluation}}, {{SemEval}}@{{ACL}} 2023, {{Toronto}}, {{Canada}}, 13-14 {{July}} 2023},
  author = {Khetan, Vivek and Wadhwa, Somin and Wallace, Byron C. and Amir, Silvio},
  editor = {Ojha, Atul Kr and Dogru{\"o}z, A. Seza and Martino, Giovanni Da San and Madabushi, Harish Tayyar and Kumar, Ritesh and Sartori, Elisa},
  year = 2023,
  pages = {2266--2274},
  publisher = {Association for Computational Linguistics},
  doi = {10.18653/V1/2023.SEMEVAL-1.311},
  url = {https://doi.org/10.18653/v1/2023.semeval-1.311},
  urldate = {2024-09-04}
}