Headstart IMDB Competition

Organized by mikesmithsheffield - Current server time: Aug. 11, 2020, 10:45 p.m. UTC

Current

Head Start
July 1, 2020, midnight UTC

End

Competition Ends
Never

Welcome!

For this competition we are using the IMDB movie reviews dataset. We are using our own data splits, so you should stick to the splits that you are given in your Google Colab.

The task is: given the text of a review, you should classify it as positive or negative. This is called a Sentiment Analysis task.

You should have attended the practical session that gives you an idea on how to solve the problem. You can, of course, change the proposed code as much as you want or even create a new project from scratch!

The only constraint is how the evaluation is done here in Codalab. Make sure you read the evaluation session!

You will need to create a Codalab account in order to participate. You can use the UoS email addresses that were given to you in order to create an account. We suggest you use the UoS account name as your user in the Codalab system, so we can easily keep track of the users. After create an account you will be able to submit as many answers as you want!

There is a Leaderboard page were your best submission will be shown. The developers of the best systems will be invited to present them during our Friday afternoon session.

We will be approving your submissions before they appear in the Leaderboard (so it may take a while for you to see your name there).

Good luck!

Evaluation Criteria

The task is going to be scored in terms of how many reviews you have correctly classified. The test data has 10,000 reviews, so your aim is to achieve 10,000!

The evaluation is done automatically and, therefore, you need to submit a file in a certain format in order to get a score. 

The format is:

  • a file called 'answer.txt' containing the predictions of your classifier, one per line, example:

positive

negative

positive

positive

...

This file should have 10,000 lines and should follow the same order as the test file, i.e. the predictions on line 1 needs to correspond to the review on line 1 in the test set (same for line 2, 3, and so on). 

  • This file 'answer.txt' should be compressed in a .zip file (called 'answer.zip')
  • You can then go to the Participate tab and submit your 'answer.zip' file
  • You can submit as many times as you want (just follow the same pipeline)

Good luck! 

Terms and Conditions

This competition is being run as part of University of Sheffield, Headstart week (Computer Science). 

There is no prizes for winner. 

Winners will be invited to describe their systems/approaches during the last session on Friday.

Head Start

Start: July 1, 2020, midnight

Competition Ends

Never

You must be logged in to participate in competitions.

Sign In
# Username Score
1 RobinS 9116.0
2 smorga41 9074.0
3 BobbyBigBrain 8786.0