Predicting Generalization in Deep Learning Forum

Go back to competition Back to thread list Post in this thread

> How to extract intermediate layer outputs?

As the models are built using a wrapped layer, traditional method of getting intermediate layer features is not working. Can you propose an alternative?

Posted by: deepq-lab @ July 30, 2020, 2:18 p.m.


Thank you for your interest in the competition. Would you mind clarifying what you mean by layer features ? A small example of what you want to accomplish and what would be the expected output would be very helpful.


Posted by: pforet @ July 30, 2020, 4:23 p.m.

Thanks for your reply, by intermediate layer features, I mean the network output at a given layer.

how do we do: clipModel = tf.keras.Model(inputs = model.inputs, outputs = model.layers[-1].output) ?
in current setup: model.inputs = [] and throws " AttributeError: Layer wrapped_layer has no inbound nodes."

Also, can you clarify how to scoring script locally? what should these paths **path/to/prediction** and **path/to/output**` point to?

Thank you

Posted by: deepq-lab @ July 30, 2020, 4:29 p.m.

Thanks for the clarification.
You can access this using some_layer._last_seen_input (this is used in the jacobian norm baseline we provided, you can refer to it for a full working example). The reason we don't use some_layer.output as you suggest it that it doesn't play nicely with the autograd, so that computing gradients with respect to some_layer._last_seen_input would work but some_layer.output would not.

Posted by: pforet @ July 30, 2020, 4:35 p.m.

thanks! this works

Posted by: deepq-lab @ July 30, 2020, 4:58 p.m.

"path to prediction" refers to the where the output of your ingestion program lies and "path to output" is anywhere you want to final score to be.

Posted by: ydjiang @ July 30, 2020, 5:48 p.m.

thanks for clarifying

Posted by: deepq-lab @ July 31, 2020, 5:23 a.m.

Hi, ydjiang. May I know whether it is possible to get the output of GlobalPooling layer in network in network model? The method 'some_layer._last_seen_input' will only get the output of layers before the GlobalPooling layer in network in network model. Naturally, we could use method 'some_layer.output', but it seems it doesn't work in TensorFlow 2.2. It would be great if you could give me some advice.

Moreover, do you know where I could find any documents about method '_last_seen_input' and 'wrapped layer'? Thanks.

Posted by: LinZhang @ Oct. 17, 2020, 3:29 p.m.

Re: Outoput of global average pooling in Nin
This is just the logit of the model. I am not sure why you would need a special method to access it.

Re: wrapped layer
This is not officially in Tensorflow. It's just a wrapper we made so people can get the input to each layer.
The code is only a couple lines. You can find it in "wrap_layer" in "" in ingestion program.

Posted by: ydjiang @ Oct. 17, 2020, 5:36 p.m.

Thanks for the response. I realize that I could add a GlobalPooling operation myself. I read the "" and it is clear. Thanks.

Posted by: LinZhang @ Oct. 18, 2020, 2:51 a.m.
Post in this thread