The Large Scale Movie Description Challenge (LSMDC) 2017 : Movie Multiple-Choice Test

Organized by atousa - Current server time: Feb. 20, 2025, 2:01 p.m. UTC

Current

Movie Multiple-Choice Test
Aug. 25, 2016, midnight UTC

End

Competition Ends
Never

 if you have any question please email to torabi.atousa@gmail.com

Overview

Natural language-based video and image search has been a long standing topic of research among information retrieval, multimedia, and computer vision communities. Several existing on-line platforms (e.g. Youtube) rely on massive human curation efforts, manually assigned tags, however as the amount of unlabeled video content grows, with advent of inexpensive mobile recording devices (e.g. smart phones), the focus is rapidly shifting to automated understand, tagging and search. In this challenge, we would like to explore a variety of different joint language-visual learning models for video annotation and retrieval task, which is based on a unified version of the recently published large-scale movie datasets (M-VAD and MPII-MD). More information about the datasets and challenge can be found here.

Multiple-Choice Test: Given a video query and 5 captions, find the correct caption for the video among 5 possible choices. The evaluation is only on public test set that we provided for multiple-choice test.

Other challenges:

  • LSMDC 2016 : Movie Description, click here
  • LSMDC 2016 :  Movie Fill-in-the-Blank, click here
  • LSMDC 2016 :  Movie Retrieval, click here

 

Participation

To participate, you should first create an account on CodaLab. In order to submit your results, please, perform these steps:

  • Convert your answer descriptions into the following JSON format:
    [
    {
    "caption": str,
    "video_id": int,
    },
    ...
    ]
  • Caption is the one of the 5 possible choices that your model pick for the test. Please provide the caption itself instead of its index. 
  • Name the JSON file as "publictest_[your_algorithm_name]_results.json" and zip it in an archive. We also provide a script to create JSON file in above format here.
  • Go to "Participate" tab, click "Submit / View Results".
  • Fill in the form (specify any external training data used by your algorithm in the "Method description" field) and upload your ZIP archive.
  • Click "Refresh Status" to see how your submission is being processed. In case of errors, please, check and correct your submission.
  • Once the submission is successfully processed, you can view your scores via "View scoring output log" and click "Post to leaderboard" to make your results publicly available.

Note, that we allow up to 10 submission a day. In total maximum submission per team is 100.

 

The evaluation is based on accuracy which is the percentage of correctly answered questions in multiple-choice test.

Winners will be selected based on a maximum accuracy of submissions on the multiple-choice test.

Movie Multiple-Choice Test

Start: Aug. 25, 2016, midnight

Competition Ends

Never

You must be logged in to participate in competitions.

Sign In
# Username Score
1 MERLOT 81.730
2 yj 78.150
3 danieljf24 75.170