I encounter such problem as well, and it will cause an value error which can interrupt the training
Posted by: team_19 @ Nov. 15, 2019, 9:21 a.m.This is likely caused by NaN's generated by your model. It's recommended to add some checking around your model outputs to make sure they are in fact floating point numbers being returned by your model and defaulting to reasonable values in the case where you receive NaN's.
Posted by: HuaweiUK @ Nov. 25, 2019, 1:34 a.m.