This page gives access to the training data and evaluation script for SemEval 2018 Task 7: Semantic Relation Extraction and Classification in Scientific Papers.
For further details please see the official competition webpage. If you are interested, consider subscribing to the dedicated google group.
Practice: 25 Sept 2017 to 7 Jan 2018. Evaluation starts on 8 Jan 2018.

Subtask 1.1 Relation classification on clean data

Subtask 1.2 Relation classification on noisy data

Subtask 2 Relation extraction and classification on clean data


  • Evaluation script for Semeval 2018 Task 7 (updated 09/01/2018)
  • This is the offline version of the scorer for Semeval 2018 Task 7.
    Usage: perl RESULTS_FILE KEY_FILE.
    Unlike in the codalab version, you don't need to specify the subtask number on the first line: the results file will be compared to the key file given as input.

    You can also evaluate your results on part of the training data using the CodaLab interface. 50 abstracts were selected for 'testing' in each subtask, you need to submit your predictions for these abstracts in a zipped file by clicking on 'Participate' on the CodaLab site. You can find the IDs of these abstracts here.