Jie Lei, Licheng Yu, Mohit Bansal, Tamara L. Berg
UNC Chapel Hill
Send emails to faq-tvqa-unc@googlegroups.com
The submissions are evaluated against various metrics as described in our paper, see the paper for details.
A valid submission file is a .zip
file containing the following 2 json files (no additional enclosing folder):
tvqa_plus_val_submission.json
: val set predictionstvqa_plus_test_submission.json
: test set predictionsThe two files are of the same format, below gives a simple example of how the file may look like, you can get more details by read the sample here. You can also see how this file is generated by looking into the baseline code for TVQAplus.
{ "ts_answer": { "141290": [[12.3, 16.4]., 2], # [[st, ed], pred_ans_idx] ... }, "raw_bbox": [ {"0": [{"word": 5297, # word id, as specified here. "pred": [0.3094744086265564, 0.33220770955085754], # prediction scores. "img_idx": 11, # image id from TVQA dataset. "bbox": [[160.25, 54.09375, 501.75, 359.25], [208.0, 5.48046875, 474.5, 359.25]], # predicted boxes associated with the word. "qid": 141862, "vid_name": "s01e02_seg02_clip_03" }, ... ], "1": [...], ..., "4": [...] }, # the keys are question-answer indices, which contains box predictions associated with the question and answer i. The program will only evaluate the one associated with GT answer. ... ] }
Before submitting, please make sure you are able to evaluate your tvqa_plus_val_submission.json
using the evaluation script here.
This page enumerated the terms and conditions of the competition.
Start: Jan. 17, 2020, midnight
Description: test evaluation
Never
You must be logged in to participate in competitions.
Sign In