Hi
Could I ask that the evaluation phase ends at midnight UTC on January 31 and why we can not submit it in 31?
Thank you.
Posted by: UoB_UK @ Jan. 31, 2021, 10:37 a.m.Hello! I have the same question. Basically, our understanding was that the competition is till the end of January 31, thus, we planned submitting our best results today. Now I see, that we misinterpreted the dates. However, if some other participants had same understanding, probably, you could give us an extra day for submissions? According to our previous participation in SemEval tasks, near the end of the evaluation phase the deadline is often moved by several days. So on behalf of all participants we are kindly ask you to give use one more chance and some time:)
Posted by: ezhick179 @ Jan. 31, 2021, 11:07 a.m.But what I understand is 00:00 on the 31st. I think there are many participants who understand this way. If you open another day, you have to notify all the participants. It is inevitable that some people will not receive the information, so there will be not fair. Besides, it seems that we can see the scores in post evaluation now, so I don't recommend to open another day for submission.
Posted by: Sattiy @ Jan. 31, 2021, 11:59 a.m.Well, during previous SemEval competitions 2-3 extra days were given after the initial deadline. The e-mails were sent to all participants, so they all could try improving their results. I don't see any unfairness here. Finally, we are not in the army to keep harsh discipline, this is a scientific competition and some additional time can make the results better and the papers more interesting - this is the final goal of SemEval as I believe.
Posted by: ezhick179 @ Jan. 31, 2021, 12:11 p.m.We can still try various methods to improve results. For now, submitting on post-evaluation, we can see the scores of every task in test dataset. This has a strong guide to the direction of model or rule optimization. That means it is some kind of data leakage for some participants who submitted results in Jan.31. It is unfair for other participants to open the submission again.
Posted by: xsysigma @ Jan. 31, 2021, 12:29 p.m.I think there is a clear misunderstanding about the deadline. The website says this.
Post Evaluation
Start: Jan. 31, 2021, midnight
Many teams including mine thought that we can submit till the end of 31st.
Posted by: tharindu @ Jan. 31, 2021, 12:37 p.m.Hi all,
Can you confirm that you can submit your datasets (today is the deadline!)? When you submit, please select "evaluation" and not "post-evaluation". All leaderboards in this competition are private.
Best regards,
The MCL-WiC team
PS. To all teams: please don't forget to fill out the submission information form (for each submission) which is mandatory in order for the submission to be accepted (today is the deadline!) You can find the link here and on our GitHub page!
Posted by: Federico_Martelli @ Jan. 31, 2021, 12:43 p.m.Thank you for your reply.
Yes, we can submit now.
Posted by: tharindu @ Jan. 31, 2021, 12:49 p.m.