In the example, it says that the answer for A. is clearly "1. to get enough sunshine" but just reading the passage suggests the reason the tree was planted was that the backyard looks empty. This gives me concern for other data in the task. Is this how all the data is assessed? Also, if I am wrong in my answer can you explain to me why it is the correct answer is 1. and not 2. for Question A.
Posted by: mill5970 @ Sept. 14, 2017, 3:43 p.m.Yes, you are obviously right: The shown example is handmade and we made an error here. To get an idea about what the actual data looks like, you can have a look at the trial data or the training and development data, which will be released next week presumably. The corpus is crowdsourced, so we will in general rely on the opinion of the crowd for what is a correct or an incorrect answer.
Posted by: simono @ Sept. 21, 2017, 1:54 p.m.Okay, that's what I figured but thank you for confirming. I looked at the trial data and everything seemed correct so I haven't found any more errors. I was looking at Task 12's trial data which had numerous spelling and logic errors before making this post so I was probably too quick to judge based on the example.
Posted by: mill5970 @ Sept. 21, 2017, 2:02 p.m.Thanks for making us aware of the mistake anyways! We are also conducting a manual validation for the final data set, which will be released shortly after, so the data should be fairly clean.
Off-topic: The notifications for the forum don't seem to work reliably, which is why your post was just spotted today, sorry for that! You can also contact us via our google group: https://groups.google.com/forum/#!forum/semeval2018-task11
Posted by: simono @ Sept. 21, 2017, 2:11 p.m.