Hi, I found that the competition using RMSE, instead of NLL or AUC.
I think Lazada need to fix this. It's a classification task, not a regression.....
Hi UdahLulus,
Thank you for the suggestion.
You are right that for binary classification task, one of the common metrics is LogLoss or NLL.
However, it is not the only possible metric. There exist multiple error metrics with different characteristics, as documented in: ttps://www.kaggle.com/wiki/Metrics/history/588. RMSE is a common general-purpose metric.
Specifically for this Lazada task, in its current formulation, it appears as a classification task. However, there is no reason why product title quality in general could only be binary. In the long run (beyond this competition), it is possible that product title quality may be rated along a scale of ratings (e.g., 0 to 5). In such an eventuality, the task may turn into regression, for which RMSE is still applicable.
In any case, I am sure Lazada would consider your point for their internal purpose. However for this public competition, the metric has been fixed to be RSME and will not be changed halfway.
Thank you.
Posted by: hadylauw @ May 6, 2017, 1:27 a.m.