DocRED

Organized by DemingYe - Current server time: Oct. 22, 2019, 5:44 a.m. UTC

Current

Challenge
June 9, 2019, midnight UTC

End

Competition Ends
Never

Welcome!

Welcome to the leaderboard for the DocRED dataset. The leaderboard displays test set results for . The data and evaluation script can be downloaded from GitHub .

Evaluation Criteria

The submissions are evaluated using relation extration F1 and evidence F1 on the test set. The evaluation script is evaluation.py is available at the Github repository. For more details, please refer to the paper.

Terms and Conditions

This page enumerated the terms and conditions of the competition.

Participantes need to compress the result file (result.json) into zip format (result.zip) for submission.

Submission file should contains all extracted relational facts from test set with necessary meta information, following the json format:

[
    {
        "title": <str>, # title of the document
        "h_idx": <int>, # index of the head entity
        "t_idx": <int>, # index of the tail entity
        "r": <str>, # Wikidata ID of the relation 
        "evidence": <list<int>> # indices of the evidence sentences
    }
]

Challenge

Start: June 9, 2019, midnight

Competition Ends

Never

You must be logged in to participate in competitions.

Sign In
# Username Score
1 Jayson 0.546
2 christoph.alt 0.543
3 pink 0.537