The ingestion output log:
========== Ingestion program version 1 ==========
************************************************
******** Processing dataset D ********
************************************************
========= Reading and converting data ==========
Info file NOT found : /tmp/codalab/tmpD9g4lS/run/input/D_public.info
The ingestion error log:
Traceback (most recent call last):
File "/tmp/codalab/tmpD9g4lS/run/ingestion_program/ingestion.py", line 215, in <module>
D = DataManager(basename, input_dir, replace_missing=True, max_samples=max_samples, verbose=verbose,testdata=0)
File "/tmp/codalab/tmpD9g4lS/run/ingestion_program/data_manager.py", line 82, in __init__
self.getInfo (info_file)
File "/tmp/codalab/tmpD9g4lS/run/ingestion_program/data_manager.py", line 356, in getInfo
self.getFormatData(os.path.join(input_dir, basename + '_train1.data'))
File "/tmp/codalab/tmpD9g4lS/run/ingestion_program/data_manager.py", line 404, in getFormatData
data = data_converter.file_to_array (filename)
File "/tmp/codalab/tmpD9g4lS/run/ingestion_program/data_converter.py", line 24, in file_to_array
with open(filename, "r") as data_file:
IOError: [Errno 2] No such file or directory: '/tmp/codalab/tmpD9g4lS/run/input/D_train1.data'
The before dataset process success, the file path is like this:
************************************************
******** Processing dataset C ********
************************************************
========= Reading and converting data ==========
Info file found : /tmp/codalab/tmpD9g4lS/run/input/C/C_public.info
https://competitions.codalab.org/forums/16559/2584/
Posted by: hugo.jair @ Aug. 13, 2018, 3:06 p.m.Hugo, I found out why this error occurs (see my message https://competitions.codalab.org/forums/16559/2594/ )
Please fix it.
Posted by: leogautheron @ Aug. 13, 2018, 4:17 p.m.Hi,
Thank you for the suggestion, but this does not fix the problem either. In either case, the public_info files are located in the same directory. Thanks anyways!
best
Posted by: hugo.jair @ Aug. 13, 2018, 4:42 p.m.The error is not on the public info files, this is on the data files.
I just got the same error. On my computer, the fix I gave you solves the issue
You are using pip version 9.0.1, however version 18.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Traceback (most recent call last):
File "/tmp/codalab/tmpoMhCOw/run/ingestion_program/ingestion.py", line 215, in <module>
D = DataManager(basename, input_dir, replace_missing=True, max_samples=max_samples, verbose=verbose,testdata=0)
File "/tmp/codalab/tmpoMhCOw/run/ingestion_program/data_manager.py", line 82, in __init__
self.getInfo (info_file)
File "/tmp/codalab/tmpoMhCOw/run/ingestion_program/data_manager.py", line 356, in getInfo
self.getFormatData(os.path.join(input_dir, basename + '_train1.data'))
File "/tmp/codalab/tmpoMhCOw/run/ingestion_program/data_manager.py", line 404, in getFormatData
data = data_converter.file_to_array (filename)
File "/tmp/codalab/tmpoMhCOw/run/ingestion_program/data_converter.py", line 24, in file_to_array
with open(filename, "r") as data_file:
IOError: [Errno 2] No such file or directory: '/tmp/codalab/tmpoMhCOw/run/input/C_train1.data'
Thanks again,
I am double checking, but I am sure this does not solve the issue in the platform (I will keep you informed)
I am curious: you said you got the same error in your local computer, are you sure is the same error? we have not been able to reproduce the error locally, thanks in advance for your input
Best
Posted by: hugo.jair @ Aug. 14, 2018, 2:48 p.m.Yes, I got the same error on my Windows computer.
My files are stored in a path like "D:\..."
and it failed for the dataset "D" because "D" is contained in "D:\..." so I had to change the code.
It does not happen every time on Codelab because this is the name of the temporary folder where the data are stored that can contain the upper case letters "B", "C", "D" or "E"
Posted by: leogautheron @ Aug. 14, 2018, 3:09 p.m.
Thanks a lot! this could certainly fix the bug: I did not get in the first place that you were referring to the tmp names. Thanks =)
Best
HJ
Posted by: hugo.jair @ Aug. 14, 2018, 3:15 p.m.