Limiting attention radius and extracting embeddings #70
Unanswered
george-henderson
asked this question in
Q&A
Replies: 1 comment
-
This is not possible with the current model
Here's an example of extracting the last hidden layer outputs from the model: #32 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello,
Is it possible to alter the model's attention radius, such that the model only applies attention within a certain window in the input?
A second question: Can you instruct on how I might extract the embeddings from the model? I am using the model out of the box, such that the final output is a tensor of dimension num_batches x input length x num_tokens, but I’d like to access the internal latent space representation of my text as well.
Thank you!
Beta Was this translation helpful? Give feedback.
All reactions