SemEval-2018 task 3 - Irony detection in English tweets Forum

Go back to competition Back to thread list Post in this thread

> Label and Score Problem

Hi,
Provided trial data contains following labels :
[Labels: **1** (ironic by clash), **2** (other irony), **3** (situational irony), **4** (non-ironic) ]
but the training data have
[* Labels: Labels: **1** (ironic by clash), **2** (situational irony), **3** (other irony), **0** (non-ironic)]

However, evaluation script following training labels. So I am following that script too. When I run the provided script with trail data on my system the results are as follows:
Accuracy:0.8
Precision:0.791666666667
Recall:0.791666666667
F1-score:0.75

But when I submit it to Coda Lab the results show 0.29 score. Please correct me if I am making some mistake.

Posted by: usmanahmed189 @ Jan. 4, 2018, 8:54 a.m.
Post in this thread