As we had mentioned earlier, we are releasing the code for the baseline system.
It is present at https://github.com/didi/iwslt2020_open_domain_translation . The README has instructions on how to set up and run the system.
The baseline is a transformer based NMT system that uses Google's tensor2tensor library.
Ja-Zh MT task Team
IWSLT 2020
Dear organisers,
Thank you very much for preparing the data and baseline systems! May I ask if there is an indicative/baseline BLEU score given your data and system configuration?
Cheers,
Patrick
Hi Patrick,
Apologies, I should have mentioned earlier.
Using the config of the baseline system just out-of-the-box (and training on the existing parallel training dataset and testing on the development dataset), we get the following best BLEU scores (char-level bleu across all different values of alpha)
Ja --> Zh : 20.03
Zh --> Ja: 27.03
This would help to sanity check your runs and build on it.
In the baseline code, we have also included the evaluation script used to generate the BLEU score.
Please let us know if you have any further questions.
- Ajay
Posted by: ajaynagesh @ Feb. 14, 2020, 1:10 a.m.Thank you very much Ajay! - Patrick
Posted by: pzchen @ Feb. 14, 2020, 1:29 a.m.