This page gives access to the training data and evaluation script for SemEval 2018 Task 7
: Semantic Relation Extraction and Classification in Scientific Papers.
For further details please see the official competition webpage
. If you are interested, consider subscribing to the dedicated google group
Practice: 25 Sept 2017 to 7 Jan 2018. Evaluation starts on 8 Jan 2018.
Subtask 1.1 Relation classification on clean data
Subtask 1.2 Relation classification on noisy data
Subtask 2 Relation extraction and classification on clean data
Evaluation script for Semeval 2018 Task 7 (updated 09/01/2018)
This is the offline version of the scorer for Semeval 2018 Task 7.
Usage: perl semeval2018_task7_scorer-v1.1.pl RESULTS_FILE KEY_FILE.
Unlike in the codalab version, you don't need to specify the subtask number on the first line:
the results file will be compared to the key file given as input.
You can also evaluate your results on part of the training data using the CodaLab interface. 50 abstracts were selected for 'testing' in each subtask, you need to submit your predictions for these abstracts in a zipped file by clicking on 'Participate' on the CodaLab site. You can find the IDs of these abstracts here.