SemEval-2019 Task 12 - Toponym Resolution in Scientific Papers Forum

Go back to competition Back to thread list Post in this thread

> Which dataset is the performance in results page based on?

Hi, Is there anyone who knows that which dataset is the performance in results page(https://competitions.codalab.org/competitions/19948#results) based on?
Sub-Task 2: Practice Toponym Disambiguation
0.8400 (1) 0.8400 (1) 0.8400 (1) 0.7759 (1) 0.7759 (1) 0.7759 (1)

I just ran the baseline system and achieve a better performance on eval data.The micro F1 is higher than 80%.

Posted by: TTCoding @ Dec. 4, 2018, 7:08 a.m.

The performances are obtained by the baseline system running on the validation corpus posted in my first post in the google group. This is the corpus yous should use during this phase to evaluate your system.

Best regards,
Davy

Posted by: dweissen @ Dec. 4, 2018, 3:43 p.m.

Thanks, Davy. Does you mean the 'Trial Corpus'(https://competitions.codalab.org/my/datasets/download/692a40c6-fd7a-4094-b2a7-f3f72e7a75f0) ? But I achieve even higher performance in this corpus( F1 is over 90%)?

Posted by: TTCoding @ Dec. 5, 2018, 3:12 a.m.

No, I mean the Validation_Data_xxx files that can be downloaded from the google group of the task 12 for all participants registered. Did you already registered to this competition? If not, please apply in codalab to this competition. I will send you an email in the following days with the instructions to complete the registration. Once done, you will be able to access the training and validation corpora and in beginning of January a link to the test corpus.

Best regards,
Davy

Posted by: dweissen @ Dec. 5, 2018, 3:20 p.m.

Thanks, Davy. I'll try it again.

Posted by: TTCoding @ Dec. 6, 2018, 6:57 a.m.
Post in this thread