Hi there!
I was wondering if there was a generic way to extract pre-activations (i.e. the input of the relu activation function) for intermediate layers?
Since the models seem to merge convolutions and activation functions in single layers, some_layer._last_seen_input will not work.
Thanks!
Simon
We are working on a solution to split them, but on the current private dataset they should be all split already.
Likewise, I believe only one task in the public dataset has merged relu's.
So for the moment you can use the one task to test your solution and submit without any problem.
Cool, thank you for your work. Much appreciated!
Posted by: simonc @ Sept. 18, 2020, 11:56 a.m.I've the following code which relies on the fact that 'Relu' appears in the name of a ReLU layer's output. I hope this will work on all model architectures:
preactivations = []
for l in model.layers:
_____if 'Relu' in l.output.name:
__________preactivations.append(l._last_seen_input)
the dataset in the final phase should be also splited(relu and conv)?
Posted by: equation @ Oct. 8, 2020, 7:06 a.m.Yes they will be split.
Posted by: ydjiang @ Oct. 8, 2020, 8 a.m.While the code described above worked on the public data, it doesn't on the private one. I got the following error for my submission:
File "/home/codalab/tmpG1r8eK/run/program/complexity.py", line 13, in collect_preactivations
if 'Relu' in l.output.name:
File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/base_layer.py", line 1827, in output
raise AttributeError('Layer ' + self.name + ' has no inbound nodes.')
AttributeError: Layer wrapped_layer has no inbound nodes.
I will delve into this to find a solution... In the meantime, any suggestions are welcome :)
Posted by: simonc @ Oct. 12, 2020, 8:50 a.m.It might be related to this issue? https://competitions.codalab.org/forums/21997/4580/
Posted by: simonc @ Oct. 12, 2020, 8:57 a.m.Hi, Simonc. Did you find how to let 'l.output' work in TensorFlow2.2? In the link you posted, it seems 'l.output' will work in TensorFlow2.3. But our submission will be run with TensorFlow2.2 online, right? Thanks.
Posted by: LinZhang @ Oct. 17, 2020, 3:18 p.m.the images in the final phase will also range in [0,1]?
Posted by: equation @ Oct. 18, 2020, 7:16 a.m.Yes
Posted by: ydjiang @ Oct. 18, 2020, 5:35 p.m.