Forums

SemEval-2019 Task 12 - Toponym Resolution in Scientific Papers Forum

Go back to competition Back to thread list Post in this thread

> About scores

Dear organizers,

Could you provide individual scores for each patient in 'View scoring output log' or 'Download output from scoring step'?
I just want to check whether there is some error in specific patient case or not.

Thanks.

Posted by: cjhsmile @ March 3, 2020, 5:09 p.m.

Hi,

Thanks for this suggestion, I think this is a really good idea! Providing a score for each patient in the output log it's definitely something we could do, but before I code it up I'm going to I'm going to double check with some of the other organizers.

Thanks,
Aaron

Posted by: OpenKBP @ March 6, 2020, 4:16 p.m.

After consulting with other members of the team we've opted not to provide patient-by-patient scores. One of the reasons is because teams could cherry pick the "best" prediction for each patient from the set of all their previous submissions. As a result, giving patient-by-patient score compromises the validity of the leaderboard.

Aaron

Posted by: OpenKBP @ March 6, 2020, 8:23 p.m.
Post in this thread