a. Does the Labels 1-5 denote the five human annotators?
b. In the evaluation task, the predicted label is matched for the similarity with the human annotators (label 1-5)?
I guess what you mean is the trial data we released before to show the format of our dataset as required by SemEval. We just released the training data for task 1 & 2, please use the new data for the competition.
In the trail data, besides the label 1 to label 5, we provided 'gold_label', and the models only need to predict 'gold_label'. Actually we kept label 1 to label 5 annotated by annotators only to make sure the agreement and we have omitted them in our newly published training data, and we only provided 'gold_label' now.