Has anyone encountered this problem？
Posted by: equation @ Oct. 22, 2020, 10:38 a.m.Hi, we are currently looking into this problems and trying to increase the disk space of our servers.
The final dataset has much more models so that might be a part of the reason the servers ran out of space (although we did not encounter this problem while testing it).
In the mean time, we can try to run this solution for you and send the results to you via email, if you are ok with that.
Hi
I am running into the same issue, the timestamp of my failed solution is "10/24/2020 19:26:25".
Should I wait for the problem to be solve and re-submit (hoping that my failed submission won't count !) ?
Thank you
Failed submissions will not count. Do not worry about it.
As long as it runs fine on the development phase, we will re-run it.
I resubmitted it myself.
Posted by: AlgueRythme @ Oct. 26, 2020, 10:52 a.m.Nvm it failed again for the same reasons... I will wait few days for the problem to be solved.
Posted by: AlgueRythme @ Oct. 26, 2020, 10:53 a.m.I submitted three times the same code already, and I am still getting the error.
Am I the only one ?
I am also hitting this error. Is it indicative that my submission is running too long? Or is this just an intermittent error? Thanks!
Posted by: yairschiff @ Oct. 29, 2020, 4:01 p.m.I think it is intermittent: after resubmitting I eventually ended having something that works. So the only solution is: keep trying.
Posted by: AlgueRythme @ Oct. 29, 2020, 4:02 p.m.@ AlgueRythme, thanks for that info. Good to know that this might not be a problem with my code.
One thing that is concerning to me though is that only my first submission provided a stack trace with the OSError. My second submission also failed but there is no stack trace and the logs are empty. Did you see the same? Does this mean I hit a new error or is it likely the same OS Error? (I believe my submission ran for about 2hrs prior to failing)
Posted by: yairschiff @ Oct. 29, 2020, 4:23 p.m.Your code failed not because the OS error. I am not sure why there is no error log on the development phase but there is an error log for the test phase.
I will send it to you via email.
@ydjiang, thank you very much for forwarding this info. Is it possible to cancel my new submissions which will presumably all fail with the same error? I created the new submissions before I. knew what the underlying cause was, because I thought it was still the same OSError
Posted by: yairschiff @ Oct. 29, 2020, 5:28 p.m.I also meet the same error that [Errno 28] No space left on device, but the same submission was correctly worked on development phase and mirror test.
The error submission was submited at 10/30/2020 20:09:07. Can you give the error log?
By the way, if the re-upload submission meet the same error after 11/1, how can we solve this issues? I am sure this submission can work normally on development phase and mirror test.
I have get the error message about the [Errno 28] No space left on device, but it seem like about zipping operation , not about our algorithm.
Traceback (most recent call last):
File "/worker/worker.py", line 335, in run
bundles = get_bundle(root_dir, 'run', bundle_url)
File "/worker/worker.py", line 195, in get_bundle
metadata[k] = get_bundle(bundle_path, k, v)
File "/worker/worker.py", line 151, in get_bundle
z.extractall(bundle_path)
File "/usr/lib/python3.6/zipfile.py", line 1524, in extractall
self._extract_member(zipinfo, path, pwd)
File "/usr/lib/python3.6/zipfile.py", line 1579, in _extract_member
shutil.copyfileobj(source, target)
File "/usr/lib/python3.6/shutil.py", line 82, in copyfileobj
fdst.write(buf)
OSError: [Errno 28] No space left on device