This page gives access to the data, evaluation script and official competition results for
SemEval 2018 Task 7: Semantic Relation Extraction and Classification in Scientific Papers.
For further details please see the
official competition webpage. If you are interested, consider subscribing to the dedicated
google group.
Practice: 25 Sept 2017 to 7 Jan 2018. Evaluation starts on 8 Jan 2018.
SemEval 2018 Task 7 Final Results
The final rankings and the detailed submission results are available:
SemEval 2018 Task 7 test data with keys
Subtask 1.1 Relation classification on clean data
Subtask 1.2 Relation classification on noisy data
Subtask 2 Relation extraction and classification on clean data
SemEval 2018 Task 7 training data
Subtask 1.1 Relation classification on clean data
Subtask 1.2 Relation classification on noisy data
Subtask 2 Relation extraction and classification on clean data
Evaluation
-
Evaluation script for Semeval 2018 Task 7 (updated 09/01/2018)
This is the offline version of the scorer for Semeval 2018 Task 7.
Usage: perl semeval2018_task7_scorer-v1.1.pl RESULTS_FILE KEY_FILE.
Unlike in the codalab version, you don't need to specify the subtask number on the first line:
the results file will be compared to the key file given as input.
You can also evaluate your results on part of the training data using the CodaLab interface. 50 abstracts were selected for 'testing' in each subtask, you need to submit your predictions for these abstracts in a zipped file by clicking on 'Participate' on the CodaLab site. You can find the IDs of these abstracts here.