Dear organizers,
is the final ranking of systems only based on pearson score? or is it the case that disk footprint size is also considered to rank systems?
best,
Shaun
Dear shaun.lim,
as stated on the webpage of the shared task:
"participants will be asked to provide info on their model size (disk space and number
of parameters) with their submission and will be able to rank systems based on that".
For this reason, the "<rank>" column of the leaderboard indicates the averaged ranking
of participants with regards to all metrics (including memory print and number of parameters).
Best,
Fred.