Introduction to Transformers
0 likes212 views
The document provides an overview of Transformers, including: - Transformers overcome limitations of RNNs by using attention mechanisms instead of recurrence. They have achieved state-of-the-art results on many NLP tasks. - Transformers use an encoder-decoder architecture, with the encoder generating representations of input text and the decoder generating output text. - The encoder and decoder each consist of stacked identical blocks containing multi-head attention and feedforward sublayers. Positional encodings allow the model to use order. - Self-attention mechanisms relate each word to every other word using query, key, value matrices, allowing the model to understand context.
Introduction to Transformers
- 1. Inside Transformers Suman Debnath Principal Developer Advocate, India
- 2. Why Transformers? • RNN and LSTM • Machine Translation • Text Generation • Next word prediction • More… • Sequential processing and learning • Challenge • Recurrent Model • Long-term dependency • Bidirectional context is not taken into account • Context understanding « The bat flew past my window » vs. « He hit the baseball with the bat »
- 3. • Overcome this limitation of RNNs • SOTA model for several NLP tasks • Paved the way for new revolutionary architectures such as BERT, GPT-3 and more • Based entirely on the attention mechanism and completely gets rid of recurrence Attention Is All You Need
- 4. Let's understand how the transformer works (language translation task) • Encoder-Decoder architecture • Feed the input sentence (source sentence) to the encoder • Encoder learns the representation of the input sentence and sends the representation to the decoder • The decoder receives the representation learned by the encoder as input and generates the output sentence (target sentence). Encoder Decoder I am good je vais bien Representation
- 5. The encoder of the transformer • Stack of N number of encoders • The output of one encoder is sent as input to the encoder above it. • Questions? • How exactly does the encoder work ? • How is it generating the representation for the given source sentence (input sentence)? Encoder Layer 1 Encoder Layer 2 I am good Representation Encoder Layer N
- 6. How exactly does the encoder work ? • All the encoder blocks are identical • Each encoder block consists of two sublayers • Multi-head attention • Feedforward network(FNN) Encoder Layer 2 Encoder Layer 1 FFN FFN Representation Multi-head attention Multi-head attention I am good
- 7. Self-attention mechanism • The pronoun it could mean either dog or food • We know, it implies the dog and not food • But how our model can understand ? • Here is where the self-attention mechanism helps us • Model relates the word it to ALL the words in the sentence “A dog ate the food because it was hungry” How exactly does this work?
- 8. I am good • The embedding of the word I is : • The embedding of the word am is : • The embedding of the word good is : *embedding dimension be 512
- 9. 3 New Matrices : {query, key, value} Q K V WQ WK WV Weight Matrices WQ WK WV (randomly initialized, learned during training)
- 10. 3 New Matrices : {query, key, value} Q K V WQ WK WV Weight Matrices WQ WK WV (randomly initialized, learned during training) Implies the query, key, and value vectors of the word “I” *the dimension of the {query, key, value} vector(dk) is 64.
- 11. Why are we computing this? What is the use of query, key, and value matrices? How is this going to help us?
- 12. Self-attention mechanism • We learnt to compute matrices : query(Q), key(K), and value(V) • They are obtained from the Input matrix, X • How they are used in the self-attention mechanism • REMEMBER: self-attention mechanism relates the word to all the words in the given sentence • This understanding helps to learn better representation • Next, we will see how these 3 matrices, helps to get a better representation
- 13. 4 Steps
- 14. Step 1 (”Dot product” between the query matrix, Q, and the key matrix, KT ) What exactly Q.KT does signify? “how similar they are”
- 15. Step 2 (”Divide” the matrix by the square root of the dimension of the key vector) Why this is needed? “useful in obtaining stable gradients” *the dimension of the key vector(dk) is 64.
- 16. Step 3 (”Normalize” the matrix by the square root of the dimension of the key vector) We normalize them using the softmax function (bringing the score in the range of 0 to 1 and the sum equals to 1) Word “I” is related to: - itself by 90% - am by 7% - good by 3%
- 17. Step 4 (final step in the self-attention mechanism is to compute the “attention matrix”) The attention matrix(Z) contains the attention values for each word in the sentence (sum of the value vectors weighted by the scores)
- 18. Self-attention of the word “I” is computed as the sum of the value vectors weighted by the scores
- 19. Using a self attention mechanism, we can understand how a word is related to all other words in the sentence. Step 1 Step 2 Step 3 Step 4
- 20. Let’s recall Encoder Layer 2 Encoder Layer 1 FFN FFN I am good Representation Multi-head attention Multi-head attention
- 21. Multi-head attention mechanism • Instead of having a single attention head, we can use multiple attention heads • Instead of computing a single attention matrix, Z, we can compute multiple attention matrices • This will be useful only in circumstances where the meaning of the actual word is ambiguous, e.g. “A dog ate the food because it was hungry”
- 22. Positional encoding • Transformer network, we don't follow the recurrence mechanism • How will it understand the meaning of the sentence if the word order is not retained ? • We should give some information about the word order to the transformer so that it can understand the sentence + =
- 23. Positional encoding -1 -0.25 0.91 -0.25 Positional Encoding for the 30th Word Dim 1 Dim 2 Dim 3 Dim 4
- 24. Let’s recall FFN FFN I am good Representation Multi-head attention Multi-head attention FFN I am good Representation Multi-head attention Positional Encoding
- 25. Feedforward network • The feedforward network consists of two dense layers with ReLU activations • The parameters of the feedforward network are the same over the different positions of the sentence and different over the encoder blocks.
- 26. Add and norm component • It connects the input and output of a sublayer. • Connects the input of the multi-head attention sublayer to its output • Connects the input of the feedforward sublayer to its output • Residual connection followed by Layer normalization • Layer normalization promotes faster training by preventing the values in each layer from changing heavily
- 28. Encoder Decoder I am good je vais bien Representation
- 29. The decoder of the transformer • Stake of N number of decoders • The output of one decoder is sent as input to the decoder above it • Decoder receives two inputs: • The previous decoder’s output • The encoder's representation • Question? • How exactly does the decoder generate the target sentence?
- 30. At time step t=1 At time step t=2 At time step t=3 Similarly, on every time step, the decoder combines the newly generated word to the input and predicts the next word. At time step t=4
- 31. How exactly does the decoder work? • All the encoder blocks are identical • Each encoder block consists of three sublayers • Masked multi-head attention • Multi-head attention • Feedforward network(FNN) Decoder Block FFN Representation Masked multi-head attention Multi-head attention
- 32. Masked multi-head attention • During training: • Since we have the right target sentence • We can just feed the whole target sentence as input to the decoder but with a small modification • During testing: • Decoder predicts the target sentence word by word in each time step • We learned that the decoder takes the input <sos> as the first token • Keep predicting the target sentence until the <eos> token is reached Training Data set
- 35. Attention Matrix Encoder Representation
- 37. Feedforward network • The feedforward layer in the decoder works exactly the same as what we learned in the encoder
- 38. Add and norm component • The add and norm component in the decoder works exactly the same as what we learned in the encoder
- 39. Linear and softmax layers • Feed the output obtained from the topmost decoder to the linear and softmax layers
- 41. The complete transformer architecture
- 42. Few great books • Resources • Data Science on AWS • Getting Started with Google BERT
- 43. Suman Debnath Developer Advocate, India /in/suman-d/