Sentence-level Post-Editing Effort QE shared task 2021 Forum

Go back to competition Back to thread list Post in this thread

> Failed Submission: what is the reason?

I receive failed submission status for Task 1 si-en, even though I formatted the output as requested on the website. Line 1 for disk, line 2 for the number of parameters, and lines 3-n+2 are for the test set scores.

What might be the reason?

Ergun

Posted by: bicici @ July 27, 2021, 9:17 a.m.


The scoring output specifies that:
###
Loading goldlabels...
done.
Loading your predictions...
done.
Computing scores...
Computed the scores in 0.0109 seconds.
Writing in output file...
----
disk_footprint:61203283968
model_params:380
pearson => better than baseline? no
----
Done!
###

The error log:
###
WARNING: Your kernel does not support swap limit capabilities or the cgroup is not mounted. Memory limited without swap.
###

The output from scoring step:
###
disk_footprint:61203283968
model_params:380
pearson:0.0610823909621
mae:2.48540940293
rmse:2.82197975289
###

Can you identify why the submission is failing?

Posted by: bicici @ July 27, 2021, 9:28 a.m.
Post in this thread