I had some issues with the team formation.Once I created the team, there are two submissions on the public leaderboard in our team's name. (One in my username and other with username and team's name). Could you please correct this?
How are submissions selected for the leaderboard? We have another submissions which gave a higher score than current score on leaderboard. Could you please let us know why this happened?
Also we have one submission which says running from more than 48 hours? Should we resubmit that or will that be re-run?
Thanks
Hi!
Thank you for your interest in our competition. It sometime happens that some submission don't show up on the leaderboard. This is not our intention as we don't filter the submissions in any way, and we are happy to fix that when it happens. Would you mind sending us an email at pgdl.neurips@gmail.com with the timestamp of your submission?
For the team creation yes, it is a known bug. I'll correct that too.
Good luck for the competition!
Posted by: pforet @ Aug. 21, 2020, 1:09 p.m.Actually I found it. I stopped it but it seems it was hanging because your function was too expensive. I've looked in your submission and there is a couple of improvements you can add to make sure it runs faster:
1) Try to use tf.function where possible. No need to put it on the functions that have low complexity, but there should probably be one where you call the model (example: predict = tf.function(model) then predictions = predict(x))
2) When computing some score, you can probably get a very good estimate by iterating on a fraction of the dataset only (maybe 5000 obs instead of the whole training set, that would make things about 10x time faster)
Let me know it that fixed it!
Hey,
Thank you for your reply and for your suggestions.
I'll surely try all the points that you mentioned.
But I'm a bit surprised that it's taking so long because I had another very very similar submission (next one) and that ran successfully. I can mail you the timestamp of the submission I'm talking about if necessary.
We had the same issues with the team formation on the leaderboard, could you please correct it?
Posted by: equation @ Aug. 24, 2020, 9:45 a.m.