Covid-19 Infection Percentage Estimation: Evaluation

Evaluation Metrics

The evaluation metrics are: Mean Absolute Error (MAE), Pearson Correlation coefficient (PC) and Root Mean Square Error (RMSE). The most important Evaluation Criterion is the MAE. In the event of two or more competitors achieve the same MAE, the PC and the RMSE are considered as the tie-breaker.

How to Participate:

• Joining the Competition: Each team should request participating to the challenge on CodaLab platform and submit the team name, the members' name, emails and affiliations to: faresbougourzi@gmail.com.

• Validation Phase: In the Validation Phase: You have to submit the predictions of the validation data as 'predictions.csv' file, that contains the name of the slice images in the first column and the corresponding Covid-19 infection percentage estimation in the second column. This file must be compressed as 'predictions.zip' file and submitted in CodaLab.

• Testing Phase: You have to submit the predictions of the Testing data as 'Team_Name.csv' file, that contains the name of the slice images in the first column and the corresponding Covid-19 infection percentage estimation in the second column. This file must be compressed as 'predictions.zip' file and submitted to: faresbougourzi@gmail.com.

• The approach description: The Top-5 teams are encouraged to submit a conference paper (that summarizes the approach and diffrent experiements) to our Workshop of Covid-19 in the 21st International Conference on IMAGE ANALYSIS AND PROCESSING (https://sites.google.com/view/covid19iciap2022/home). Otherwise, send us the ARXIV link.

Competition Schedule

• 16 October 2021: Release of the Train and Val Sets.
• 20 December 2021: Release of the Test Set.
• 30 December 2021: Deadline to Submit the Test Predictions.

Competition Rules

1. Team size: Teams should consist of a minimum of 2 and maximum of 7 members.
2. Open Source:   The workflow, codes, and presentations created as part of the competition will be as open source.
3. General Rules: Participants should estimate the percentage of Covid-19 infection from each slice using Machine Learning. Only ImageNet's pre-trained models and Lung Nodule Segmentation models are allowed. The use of external data or other pre-trained models is not allowed. The models must be trained using the training data and evaluated using the validation data.
4. Citation: The provided dataset is available for research purposes only. If you use this dataset, you should cite it appropriately and not use the work for commercial purposes. In particular, you should cite the following works:
   
   
   @Article{jimaging7090189,
   AUTHOR = {Bougourzi, Fares and Distante, Cosimo and Ouafi, Abdelkrim and Dornaika, Fadi and Hadid, Abdenour and Taleb-Ahmed, Abdelmalik},
   TITLE = {Per-COVID-19: A Benchmark Dataset for COVID-19 Percentage Estimation from CT-Scans},
   JOURNAL = {Journal of Imaging},
   VOLUME = {7},
   YEAR = {2021},
   NUMBER = {9},
   ARTICLE-NUMBER = {189},
   URL = {https://www.mdpi.com/2313-433X/7/9/189},
   ISSN = {2313-433X},
   DOI = {10.3390/jimaging7090189}
   }
   
   
   @article{vantaggiato2021covid,
   title={Covid-19 recognition using ensemble-cnns in two new chest x-ray databases},
   author={Vantaggiato, Edoardo and Paladini, Emanuela and Bougourzi, Fares and Distante, Cosimo and Hadid, Abdenour and Taleb-Ahmed, Abdelmalik},
   journal={Sensors},
   volume={21},
   number={5},
   pages={1742},
   year={2021},
   publisher={Multidisciplinary Digital Publishing Institute}
   }

The Organizers:

• Dr. Fares Bougourzi: Institute of Applied Sciences and Intelligent Systems, National Research Council of Italy, 73100 Lecce, Italy.

• Pr. Cosimo Distante: Institute of Applied Sciences and Intelligent Systems, National Research Council of Italy, 73100 Lecce, Italy.

• Pr. Abdelmalik Taleb-Ahmed: Univ. Polytechnique Hauts-de-France, Univ. Lille, CNRS, Centrale Lille, UMR 8520 - IEMN, F-59313 Valenciennes, France.

• Pr. Fadi Dornaika: University of the Basque Country UPV/EHU, San Sebastian, Spain; IKERBASQUE, Basque Foundation for Science, Bilbao, Spain.

• Pr. Abdenour Hadid: Univ. Polytechnique Hauts-de-France, Univ. Lille, CNRS, Centrale Lille, UMR 8520 - IEMN, F-59313 Valenciennes, France.

Website:

• For More Details: https://github.com/faresbougourzi/Covid-19-Infection-Percentage-Estimation-Challenge

• Contact: faresbougourzi@gmail.com; abdelmalik.taleb-ahmed@uphf.fr