Version 1.0 contains 2010 annotated sentences that describe binding interactions between various biological entities (proteins, chemicals, etc.).
1405 sentences are for training, another 201 sentences are for validation. They are publicly available at https://github.com/YerevaNN/BioRelEx/releases
Another 404 sentences are for testing which are kept private for this Codalab competition. All sentences contain words "bind", "bound" or "binding".
For every sentence we provide:
The goal of this challenge is to learn end-to-end relation extraction from raw sentences.
Read more about the benchmark and the baselines in our paper at ACL Anthology.
The main evaluation metric is F-score of the extracted relations. We additionally report precision and recall. If the same entity is mentioned multiple times in the sentence (e.g. synonyms or abbreviations), we expect to the models to output relations between all possible versions.
We do not have an explicit measure of coreference detection. It is implicitly captured in the relation extraction measures.
The evaluation ignores the labels of relations (positive, negative and neutral are all merged into a single class). It also ignores entity types and grounding information.
We also report precision, recall and F-score for entity mention detection. If the same entity is mentioned multiple times, we count all mentions of the entity (e.g. all synonyms and all mentions in the sentence).
The evaluation script is available at https://github.com/YerevaNN/BioRelEx
Read more about the evaluation and the baselines in our paper at ACL Anthology.
Sample submission is available at https://github.com/YerevaNN/biorelex-codalab-submission
Start: July 1, 2019, midnight
Description: Development phase
Start: July 1, 2019, midnight
Description: Test phase
Dec. 31, 2050, midnight
You must be logged in to participate in competitions.
Sign In# | Username | Score |
---|---|---|
1 | anonml1 | 67.09 |
2 | tuanlai245 | 62.18 |
3 | mahnerak | 50.15 |