OpenKBP - 2020 AAPM Grand Challenge Forum

Go back to competition Back to thread list Post in this thread

> Confusion about the evaluation code

In provided_code/
line 127:
fractional_volume_to_evaluate = np.maximum(100, 100 - voxels_in_tenth_of_cc/roi_size * 100)
metric_eval = np.percentile(roi_dose, fractional_volume_to_evaluate)

Doesn't it mean “fractional_volume_to_evaluate“” is always equal to 100?

Posted by: LSL000UD @ March 22, 2020, 7:41 a.m.

I'm very sorry for any confusion. This was a bug that should have a very minor impact on everyone's DVH scores (no one's ranking changed). We have fixed the evaluation script on CodaLab, which had the same bug, and we are reevaluated all the leaderboard scores.

We have also fixed the evaluation function on GitHub so that new participants won't be affected. We just removed the np.maximum function in line 127 of provided_code/ That is:

line 127:
fractional_volume_to_evaluate = 100 - voxels_in_tenth_of_cc/roi_size * 100

Thank you for very much for catching this bug. Please let me know if you have any questions.


Posted by: OpenKBP @ March 22, 2020, 6:03 p.m.
Post in this thread