Forums

Sentence-level Direct Assessment QE shared task 2021 Forum

Go back to competition Back to thread list Post in this thread

> Evaluation Metrics Question

Hi, May I know what the "rank" means in the metrics, except Pearson, MAE, and RMSE ? Thank you.

Posted by: joanne.wjy @ July 20, 2021, 12:56 p.m.

Dear joanne.wjy,
the leaderboard indicates, for each metric individually, the ranking of a participant with regards to the other
submissions (e.g., (1) means 1st, (2) means 2nd, etc.).
The "<rank>" column of the leaderboard simply indicates the averaged ranking of a participant with
regards to all metrics (including memory print and number of parameters).

Best,
Fred.

Posted by: fblain @ July 20, 2021, 1:24 p.m.
Post in this thread